Managing the privacy and security complexities of open data


April 3, 2017

Illustration of a method of DNA sequencing. Image with depth of field.

A decade ago, Clive Humby — a data science innovator — made headlines when he declared that “data was the new oil”. It is a great metaphor that has even more relevance today because it underlines the potential value of data, when refined, to unlock innovation and productivity across our economy.

Indeed, more technology leaders and governments are labelling data as the currency for growth and organisations across Australia are sitting on a gold mine of unrefined data.

Yet when the Productivity Commission’s final report is revealed, one of the key challenges to be solved is the inherent privacy and security complexities of opening up datasets to fuel innovation without compromising individual, government or business interests.

As Co-Founder of Data Republic, I agree that trust should be the foundation for open data to reach its full potential and this might be achieved with an exchange model rather than a highly regulated approach.

I can say that we spent a lot of time building a system based on trust that proactively addressed issues of governance, privacy and security. In fact, our first year of operation was spent working closely with Allens Legal team to develop a legal framework to allow organisations that collect data to securely exchange it.

Undoubtedly the Commission will have its own views and recommendations when it comes to tackling these issues, but having already built a legal framework which solves the governance, privacy and security challenges of data exchange, I want to flag our learnings for the government to consider.

Privacy protection

Consumer privacy has to come first if you are to engender trust in a data sharing framework.

De-identified data is our default position — every effort should be given to reduce risks to re-identification of individuals without their express consent.

For us this meant mandating that only de-identified data could be exchanged on our platform and where it was required to refer identifiers, that all personal information related to an individual was replaced by anonymous ‘tokens’ from data owners before data was ingested into the exchange environment.

We’ve also created various security and re-identification controls which we apply to every analysis conducted on our platform and run stringent checks on organisations contributing data onto our platform to ensure that they have the applicable privacy policies, data security and governance procedures.

The draft Productivity Commission report raises important points about consumer privacy. And as we move towards greater accessibility to and greater liquidity of curated datasets, the next step will be to consider the need to develop actionable and clear policies and guidelines for businesses and government departments to protect the consumer when opening up and sharing data.

Ecosystem effect

Data exchanges are governed by the Australian Privacy Principles Guidelines (APP), requiring that privacy law specialists review before, during and after solutions to ensure the data exchange process meets expectations.

In addition to abiding by these guidelines, organisations typically go through an additional layer of legal and commercial negotiations which define what data will be shared and for what purpose. The negotiations can be complicated as they define both the potential value and risk-profile of exchanges.

As a result of this complexity, data sharing agreements are often developed in an ad hoc manner or one a ‘one-off’ basis between organisations, with separate agreements drafted each time– a process that can be both costly and time consuming for organisations.

Within its legal framework and platform, Data Republic developed an opt-in ‘private by design’ model that allows companies to transparently negotiate multi-lateral data sharing agreements, ensuring that all parties adhere to the same rules without the time-consuming and repeated legal process.

By defining and enforcing a shared set of principles or ‘rules’ for each organisation within our data exchange ecosystem, we create a level-playing field for participants who can get productive faster and negotiate safe, privacy complaint data exchanges within a matter of hours — not months, because the right legal framework is already in place to protect companies and consumers.

Only when data sharing agreements can be made both safely and at scale, can we really see the productivity and innovation benefits of data exchange.

Data innovation in a demilitarised zone

To ensure the liquidity of data and its ability to boost economic productivity — from encouraging operational efficiencies to identifying gaps for innovation, boosting competition with expanded products and services, and potentially carving out market opportunities for entirely new business models is preserved — datasets should be available to be innovated on again and again without the risk of patent trolling on particular types of datasets.

A key element of Data Republic’s platform has been the creation of a demilitarised zone to prevent innovation from being stifled. Data Republic’s platform and legal framework allows Intellectual Property to be created by participants but not enforced on the platform. In this way, the potential for an individual dataset can be unlocked because it can be accessed and used by multiple companies, partners and analysts even if it has been used to create a particular report or product for one specific problem.

In-built risk mitigation and governance controls

Governance is one of the most significant issues to be dealt with to guarantee that organisations have a full audit trail on what data is leaving the organisation, which organisations are receiving the data, how the data is being used and the people who have access to it. This will be a crucial factor for governments as they look towards open data policies and enabling selective sharing of data.

To this end, various risk mitigation and governance controls should be built into a framework with the flexibility to control every aspect of the exchange process and providing different layers of access to users based on their requirements and ‘trust credentials’ within the data exchange ecosystem.

As an example, a standalone analyst working on-behalf of a retail store may have different permissions around dataset types than a medical researcher.

Importantly, no matter what the user-type, on our platform controls have been put in place to allow for a full audit trail around data analysis activity so that that usage and manipulation of data can be monitored and compliance with the legal agreement and ethics charter is ensured.

The reason this type of system works — and why we believe it should be considered by the federal government — is because it is based on a system of trust between users and a shared, recognised responsibility to adhere to the legal framework which not only protects the privacy of individuals but the rights of participants to open innovation using data. As the Productivity Commission’s draft report concluded, a lack of trust is one of key reasons why we are not maximising the value of Australia’s data, and developing that trust in a system of safe data sharing is one of our countries greatest global opportunities.

Paul McCarney, CEO and Co-Founder of Data Republic, a data exchange technology platform and marketplace for listing, exchanging and collaborating on data projects.

About the author
Inline Feedbacks
View all comments

The essential resource for effective
public sector professionals