Government bodies need to take a special approach to procurement when it comes to artificial intelligence, according to the World Economic Forum’s Centre for the Fourth Industrial Revolution.
Tried and tested public-sector procurement processes might not be suitable due to new and unique risks to manage and the possibility of unintended consequences, according to the global network focused on technology governance. It favours new approaches that support iterative development, multi-disciplinary teams on both sides of the procurement equation, and a level playing field for suppliers.
“Use procurement processes that focus not on prescribing a specific solution but rather on outlining problems and opportunities, and allow room for iteration,” recommends the international body in a white paper proposing a set of 10 AI-purchasing guidelines for public servants which are being piloted by the UK government.
“Define the public benefit of using AI while assessing risks,” is the second tip.
The guidelines urge public-sector procurement teams to be clear with potential suppliers about why they see AI as the right solution, but be open to other options. They should make sure potential bidders know “public benefit” is the aim of the game, and always conduct risk and impact assessments — both at the start of the process and after key decisions are made.
The white paper suggests agencies should collaborate and learn from each other’s approaches to AI procurement and try to align their purchasing processes with any relevant “national strategies” to do with innovation and emerging technology.
AI procurement processes need to be supported by data governance frameworks. Agencies are encouraged to be clear on where the data is going to come from, and how it will be managed, including whether it will be shared with the supplier. They should also consider and highlight any potential problems or limitations of the data that might cause bias or otherwise affect the application of AI, along with their plan to deal with these and any other unforeseen data issues.
The WEF guidelines also encourage agencies to: “Focus throughout the procurement process on mechanisms of algorithmic accountability and of transparency norms.”
This means decision-making should be as transparent as possible and agencies should promote “a culture of accountability across AI-powered solutions” as well as exploring ways that the workings of algorithms can be interpreted and contested by people inside and outside government.
The guidelines also recommend that agencies maintain ongoing relationships with AI providers involving knowledge transfer to the government body. “Ask the AI provider for insights on how to manage the appropriate use of the application by non-specialists.”
But why? There are several reasons to develop a new approach to procurement specifically for AI, according to the publication. “First, government and the general public have justified concerns over bias, privacy, accountability, transparency and overall complexity.”
Potential “negative consequences” of AI and related technologies are becoming increasingly clear and public officials must take them seriously.
“Governments do not have the latitude of using the inscrutable ‘black box’ algorithms that increasingly characterize AI deployed by industry,” the guidelines note. They must provide “accountability, transparency and explainability” to maintain public confidence and avoid creating new risks or causing harm. They are not going to be developing the systems themselves so they must communicate these needs to potential suppliers.
The authors suggest special guidelines for AI procurement are a natural extension of newer approaches to procurement of ICT products.
“Established principles of good government technology procurement may take on added significance in AI procurement. For example, many governments already ensure that procurement efforts are run by multidisciplinary teams.
“Experience has shown that a lack of diversity in AI teams and positions of leadership has correlated with inadvertent harms or discrimination to vulnerable minority groups and protected classes. Given government’s role in upholding inclusion, an added emphasis on a multidisciplinary approach and diversity may be necessary in AI procurement.”
With AI and related technologies advancing in leaps and bounds, governments are bound to keep finding new products that are of use to them but which come with risks along with the benefits. “It is important that governments prepare for this future now by investing in building responsible practices for how they procure AI,” argue the authors.
“Finally, government procurement rules and purchasing practices often have a strong influence on markets, particularly in their early stages of development. As industry debates setting its own standards on these technologies, the government’s moral authority and credibility can help set a baseline for these discussions.”