I met data at work. Data had spent much time in the West showing exemplary results for public sector innovation. When data came to my home country - India, there was intrigue and some implicit demand.

As someone who worked at the intersection of regulation, startups, and private sector, I had heard of data before. Data was smart, incisive, scientific and unlike other ‘innovation’ things – factual. Unlike others I had met, data was also verifiable and accurate – which made data really one of a kind. The folks in the governments would love data. It seemed like a good match.

My first project focused on leveraging open data for better urban mobility. Data could inform policy making and hold public sector accountable. It could also bring forth new business models generating fresh capital and support from the private sector. It sounded like panacea.

It did deliver but like every relationship, it needed to be handled and managed well.

When the Accelerator Lab Pacific first began its collaboration with a public sector agency in one of our client countries, it was their dataset that enamored us. We saw great potential in how their data could be used by communities to chart their development planning. We could further crunch the dataset for negative and positive deviance – identify those who did exceptionally well or badly and begin to understand the conditions that led to that deviance. Furthermore, we could accelerate the government’s appetite to leverage data for decision making.

The wheels were set in motion. We partnered with their team to build capacity, brought in technological expertise that could assist us and finally analyzed the information for greater insights.

However, like all relationships, the road to ‘happily ever after’ was rocky. What I learnt will form the cornerstone of our future engagements.

Walking a mile in the shoes of the data owner

Public sector actors are data owners but also regulators. They are skeptical of its use and worry about the possible backlash due to data sharing. This compels them to regulate its access and use, the most common way being to prohibit the sharing of data. In order to build trust and a long-term partnership with public sector partners, we need to acknowledge challenges and build for safeguards. Using existing UN Global Pulse’s Privacy and Data Protection Principles and similar guidelines can ease the process of working together. This can enhance buy-in and build capacity and appetite for data analysis especially in cases where there is little or poor precedence.

Dipping the toe in water vs. jumping in the pool

Data is an amorphous, vague term which refers to anything and everything. Data could vary in size from 5 lines to of 50,000 lines or in form - structured, voice, images. For data to produce insights, it needs to be organized, cleaned, and filtered for relevance and timeliness. This is time consuming and unwieldy and does not inspire public administrations to dedicate resources on data especially since they are very often under-resourced. This leads to failure in prioritizing data deeming it redundant and obsolete. The more obsolete the data, the harder it is to use. One way of breaking this reinforcing loop is starting small and building a proof of concept. In order to showcase the opportunities that good and conscious use of data can produce, we do not ask for an entire dataset. We can start with a smaller dataset instead of going big. By starting small, we can demonstrate results through quick experiments. This in turn would lay the foundation for a longer and sustained engagement.

If data is analyzed and insights never shared, does the data exist all?

One of the significant outcomes of data experiments and innovation is data-informed decision making. While this sounds sensible on paper, the journey from data points to insights is a long one. While it has as much to do with the techniques and questions we are asking of data, we cannot ignore the ethical aspects of data sharing. We need to minimize the risks of harms and maximize the positive impacts through clear guidelines on data access, use and sharing. We need to engage our government partner in the process to enable a relationship of transparency and trust. This is key because without their direction, intent, and commitment, we will never be able to leverage data optimally. This is also the precursor to moving towards an open data framework where multiple stakeholders, especially citizens could use the data for better civic planning building appetite for participatory governance and citizen science.

Like all good marriages, working with data has not been straightforward and has had significant challenges. But getting to know data better has also made life significantly better making it totally worth it.