Beware the busywork: auditor’s call for outcomes, not activity quotas

By David Donaldson

Friday February 10, 2017

They say that what gets measured gets done, and a recent liquor regulator’s audit demonstrates how process-based reporting for regulators can cloud an agency’s focus on outcomes.

The Victorian Commission for Gambling and Liquor Regulation was created to bring an integrated and risk-based approach to regulation of the two risk-prone industries, yet the VCGLR has struggled to implement a risk-based approach, with the Victorian Auditor General’s Office finding its approach inadequate, unreliable, naïve and missing the point:

“Compliance activities are not sufficiently risk based because VCGLR has focused on meeting a target number of inspections, rather than directing inspections to where noncompliance has a high risk or high potential for harm. This approach to compliance does not support the legislative objectives for harm minimisation,” says the auditor’s report on the commission, released this week.

Publicly-reported performance data from the regulator provides focuses on activity, providing limited insight into its effectiveness in meeting legislative objectives for harm minimisation, the auditor argues. This emphasis on counting activities such as the number of compliance inspections “has encouraged operational behaviour that focuses on matters with little relevance to, or impact on, harm minimisation.”

Because it does not effectively prioritise resources based on risk, proper scrutiny is not brought to bear on things like liquor licence applications by high-risk individuals. Worryingly for a regulator in such corruptible industries, the commission “largely accepts the information provided to it by these applicants at face value”, relying on the honesty of applicants and the ability of police and others to raise objections.

But this system is not working very effectively — the auditor was able to identify examples where licences were granted “without fully identifying and assessing the suitability of applicants and their associates, including cases where applicants had not provided complete information on their associates and past criminal convictions.”

There are problems with record-keeping and the conduct of compliance inspections, too. VAGO believes data on liquor licence compliance activities is “not reliable”. An internal review in 2015 found that for 22 of the 50 liquor inspections tested, the compliance team was unable to provide the internal audit team with evidence of the inspection being performed. In one example, an analysis of the data showed an inspector attending up to three inspections in different suburbs at the same time.

That review also found that 8%, or 1100, of all liquor inspections undertaken in 2014–15 were external observations — inspecting the venue from the outside without physically entering the premises. “The purpose of these external inspections was not always clear and they often only lasted a few minutes,” says the auditor.

The commission’s compliance monitoring problems arise from “longstanding serious and systemic weaknesses in the design and operation of its compliance activities,” VAGO argues. Key issues include:

  • inflexible allocation of resources to compliance activities based on factors other than risk;
  • a management approach and culture focused on meeting quotas, which encourage superficial inspection activities rather than activities to address harms;
  • inadequate guidance and training for inspectors;
  • unreliable data about liquor and gambling inspections.

Post-merger woes

A budget cut of 30% over the four years from 2012, when the commission was created by merging two agencies, has not helped — 46 of the original 287 or so staff have taken redundancies since the merger.

To make matters more difficult, the commission inherited some ongoing management and engagement issues from its predecessors:

  • the second lowest staff satisfaction levels in the Victorian public sector, as measured by the 2012 People Matter survey;
  • 12 industrial and employee relations matters carried over from RAV, including performance management cases and a serious bullying case;
  • dissent between compliance inspectors and management at the time of VCGLR’s establishment over the decision to bring in inspectors from RAV and VCGR at different pay levels and working conditions.

The VCGLR also lacked a single IT platform across its liquor and gambling activities, maintaining 30 legacy systems until the launch of its single solution in December 2016.

VCGLR chief executive Catherine Myers conceded to The Mandarin in an interview last year that merging two separate offices “wasn’t simple” and was still a work in progress.

“Change takes time,” she explained.  “You can read lots of articles around mergers and acquisitions and it’s three to five years to really build that culture and build that forward plan so people have the same vision and the values of the organisation.”

A review by the chair of the VCGLR in late 2015 identified a range of problems with leadership instability, workplace culture, siloed functions, poor systems and resourcing issues. More stable leadership and ongoing work to deal with these problems are encouraging despite much more remaining to be done, says the auditor:

“VCGLR’s plans and actions to further develop its risk-based approaches to licensing and compliance are largely sound, and its recent focused attention to improving the way it manages, develops and deploys its regulatory staff, particularly compliance inspectors, is encouraging. However, these actions are not yet complete and the scale of required reform is significant, meaning that much work remains for VCGLR to become a fully effective regulator.”

The auditor-general’s office issued 14 recommendations for the VCGLR, Victoria Police and the Department of Justice and Regulation, which have all been accepted. VCGLR provided a detailed action plan on how it will address the recommendations, noting however that its ability to fully action all recommendations is dependent on the availability of funding.

About the author
Inline Feedbacks
View all comments

The essential resource for effective
public sector professionals