Leveraging non-traditional data

Data is now recognized as ‘the new oil’ for the digital economy. While development actors have relied on traditional data sources such as public surveys and government administrative data, there is great potential to harness the value of unconventional or non-traditional data from the private sector, which can help fuel a more nimble, agile and inclusive governance.

Private companies routinely collect, analyze and use large volumes of data to derive actionable insights and inform business strategies. The ability and pace at which this data is harnessed with the aid of data science, analytics and artificial intelligence tools has allowed businesses to successfully navigate through several forms of crisis, including the COVID-19 pandemic. In this dynamic and uncertain environment, the importance of high-frequency, timely and granular data to inform decision-making has become invaluable.

Can we harness the power of data for the public good? Can we bridge the data gap to give governments access to data, insights and tools that can inform their response and recovery strategies?

There is increasing recognition that traditional and non-traditional data should be seen as complementary. Non-traditional data can bring significant benefits in bridging existing data gaps but must be calibrated against benchmarks based on established traditional data. These traditional datasets are widely seen as reliable, being subject to established stringent international and national standards. However, they are often limited in frequency and granularity, especially in low- and middle-income countries.

Meanwhile, non-traditional data such as market research routinely collected monthly from nationwide household surveys may only be specific to certain products and brands, but can provide more frequent and granular information, with disaggregation by geographical area, socio-economic group of households, gender and other attributes. Data collected from mobile devices, internet platforms and satellite images are available in real-time and offer high granularity in location.

UNDP, with support from The Rockefeller Foundation and the Government of Japan, setup the Pintig Lab: a multidisciplinary network of data scientists, economists, epidemiologists, mathematicians and political scientists tasked with supporting data-driven crisis response and development strategies. In 2021, the Lab conducted a study on how data on household spending on consumer-packaged goods can been used to assess the socioeconomic impact of COVID-19 and identify heterogeneities in the pace of recovery across Philippine households. The National Economic and Development Authority  is incorporating this data for GDP forecasting, as additional input to their predictive models for consumption. Further, this data can be combined with credit card or mobile wallet transactions and machine learning techniques for higher-frequency GDP nowcasting, to allow for nimble and responsive economic policies that can absorb and anticipate the shocks of crisis.

Non-traditional data has the potential to provide insights on the status of vulnerable groups, including the informal sector, which are not fully captured by official statistics. As such, the Department of Information and Communications Technology and UNDP are exploring the use of satellite imagery to identify ‘last mile’ communities and understand their level of connectivity in terms of WiFi, electricity, roads, education, health care and markets. UNDP has utilised chatbots on social media platforms to rapidly collate information from disadvantaged sectors and small enterprises, to understand how the pandemic has impacted them and the extent to which the social amelioration programs have worked.

These are powerful examples on how non-traditional data can and has shed light on disadvantaged groups previously invisible, allowing for more inclusive programming so that no one gets left behind.

We have only begun to open the door to a parallel world of non-traditional data that has existed alongside us for decades now. Data is inherently political and maximizing its positive impacts for society will require a concerted effort from all stakeholders to shape the ways in which data is accessed, analyzed and used beyond the confines of their “for-profit” origins. Doing so could unlock the potential for more rapid and inclusive evidence-based interventions for those who need it the most.

*      *      *

Dr. Selva Ramachandran is UNDP Philippines Resident Representative.

Learn more at ph.undp.org or follow at @UNDPPH.