Authors: Adriana Alvarado y Jorge Munguía

As COVID-19 positive cases appeared in Mexico City[1] and the effects of the pandemic were becoming visible in multiple dimensions —beyond the overwhelmed healthcare system—citizens reached out to each other to understand, assist, and reassess expectations for work, education, provision of goods, wellbeing, and care.

Although the connections and exchanges between individuals are difficult to track via statistical data, they are key to understand how social ties and citizen initiatives can best support a community’s response to the effects of the COVID-19 pandemic. But how could we learn more about these community initiatives, their effects, and about where they were being implemented? What barriers and unexpected challenges were they facing? Can a strong network of social ties boost the initiatives’ positive effects? And if so, how?

As research indicates, social capital has a major role in crisis reaction and recovery, and that is why the AccLab wondered how social capital may inform us about Mexico City’s initial reaction to the pandemic. Read more about this in this previous blogpost. Our approach began by identifying social capital at municipal and neighborhood levels in four dimensions: bridging, linking, bonding, and inequality, using existing statistical data available. In parallel, using surveys and interviews, we explored citizen initiatives in reaction to the pandemic and their experiences during the first few months.

While these approaches were direct, they also had limitations. For the social capital indicators, we worked with statistical data available that was not all recent, building tangential approximations to our subject. Furthermore, as the crisis unfolded, there were no official sources of data that portrayed citizens’ needs and strategies to address and innovate the challenges imposed by COVID-19 pandemic. In relation to initiatives, the surveys were limited in reach by our own capacity to identify and connect with a broad scope of citizen initiatives. We wondered if ongoing conversations on social media could inform us about how initiatives relate to one another, to communities, and to government programs. We began to analyze Twitter data as an alternative source of narratives that could reflect unusual citizen strategies to address the COVID-19 crisis. Social media data has been widely used in the past to understand people's perception about local crises [2], trends in discriminatory practices [3], online misinformation [4], just to mention a few.

In Mexico City, Twitter is a rich environment for the examination of social practices within the digital sphere [5]. According to recent statistics, there are eleven million Twitter users in Mexico, representing 60% of Internet users between 16 and 64 years old [6].  Additionally, Twitter allows for data collection. Specifically, we expected to inform our understanding of the geographical distribution of initiatives and learn about their actions.

Process | What We Did

To guide our Twitter research, we first analyzed responses to one of the open-ended questions from our survey, which asked for a description of the citizens' initiatives. The 147 responses offered rich narratives of the initiatives’ objectives and target population. From this analysis, we compiled a list of hashtags, keywords, and social media accounts of people and organizations that coordinated initiatives. This information was organized based on the categories of the initiatives' purpose. Table 1 shows a sample of the attributes we collected for initiatives that focused on food. 

Table 1. A sample of hashtags, keywords, and text description extracted from the initiatives collected through the survey. The first row shows the original text in Spanish, and the second row shows the translation in English.

Our second step was to collect data from Twitter. Using the Twitter API, we collected data between February 28th and May 17th, 2020, corresponding to the initial epidemiological phases of COVID-19 in Mexico City. We filtered the search of tweets using the name of the sixteen alcaldías, or municipalities, of Mexico City in combination with the hashtags and keywords collected in the preliminary analysis. In total, we collected 300,361 tweets.

A natural language processing (NLP) tool for Spanish was then used to support the analysis of the collected data from Twitter based on word embeddings using the word2vec algorithm [7].

The input for our tool was a set of cleaned tweets from each municipality and a set of sentences describing an initiative. Cleaned tweets and initiative sentences were mapped to vectors using word2vec with SBWCE. Then, each tweet vector was tested for cosine similarity with each initiative's sentence vector. In this context, a cosine similarity close to one indicates a tweet was semantically similar to a given initiative's description sentence. We used NLP to reduce the overhead of manually analyzing this relationship. 

We obtained a single output file per municipality with approximately 1000 and 1500 tweets with content related to initiatives. Using NLP helped us to narrow the number of tweets we analyzed in the following stages. The next stage of analysis consisted of further reducing the number of tweets for qualitative analysis. To this end, we selected a random sample of 100 tweets per municipality, and considering the definitions of bonding, bridging, and linking, we categorized each tweet into one type of social capital. After the first round of analysis, we continued sampling tweets in batches of 100 until we reached saturation of each municipality, meaning that no additional evidence was found.

Figure 2. Tweets collected and selected for further analysis by municipality.

As we associated the tweets, we gathered additional information that we used later to guide our interpretation. In the last stage, we used content analysis to identify commonalities, distinctions, and relationships amongst the local responses that citizens, governments, and grassroots efforts coordinated in each municipality to address the COVID-19 crisis. Our emerging themes reflected the different manifestations of linking and bridging.

Findings

Incorporating Twitter into our research revealed the potential of social media data analysis in the context of development projects as a valuable platform to examine situated knowledge that would be difficult to capture otherwise. In Mexico City, Twitter’s widespread adoption among initiatives made it a powerful source to identify civic responses that contributed to the city's recovery process.

However, due to the locality and uniqueness of some of these responses, we were only able to register the initiatives after they were posted in the public sphere of Twitter.

Around 60 ejidatarios (common land shareholders) from the town of San Gregorio, in Xochimilco, got together to deliver their products to whoever requests it in Mexico City. Here you will find the number to order:

Buy your errands with products from the chinampas and support the farmers of Xochimilco.

"De la chinampa" is a group of farmers from Xochimilco that markets their products online. Currently they continue to distribute their products, check the delivery dates.

Table 2. A sample of tweets referring to Xochimilco farmers’ initiative.

Through our analysis of Twitter data, we also experienced limitations of accuracy and representation.

1. Accuracy: since social media data are unstructured and difficult to verify, we had to develop mechanisms to ensure the quality and trustworthiness of the data. For example, we corroborated the existence of initiatives we found by searching for additional information online.
2. Representation within data: the data collected from any social media platform will always be biased due to the demographic sample, limiting whose perspectives and needs are visible for analysis.
   

We recognize that data from social media platforms are always incomplete and will never be exhaustive of the citizens and government experiences when addressing a crisis. However, they provided evidence of collective responses, community assets, and particularities of each municipality that otherwise would have been difficult to identify. Considering the limitations and constraints of Twitter data, we decided that the most appropriate approach to make sense of the data collected was to contribute to a more holistic understanding of each municipality.

Our final analysis consisted of an individual qualitative overview of each municipality and a comparison of its strategies and priorities. We characterized each municipality by summarizing individual observations on linking, bridging, and context characteristics. The characterization of linking consisted of a description of local government strategies when responding to populations' needs, the populations that were a priority for each municipality, and the communication between citizens and government. Bridging consisted of the collective efforts of communities who organized and leveraged their resources to address the social and economic consequences of the pandemic. Lastly, context characteristics consisted of describing local problems rooted in the municipalities. The data from Twitter allowed us to identify people, local assets, narratives, rhythms, and organization models that otherwise would have been difficult to find. This initial exploration uncovered the richness of approaches and the plurality of ways of knowing and doing in times of crisis.

Towards new applications

Our findings show that social media data and other digital traces are a complementary source to statistical data and act as a window to understand the narratives in the public sphere. However, it remains to be explored how to integrate uncommon sources of data into the more traditional practices of development work. Below we outline a set of recommendations to be explored in the future: 

1. Any evidence obtained from user-generated content should be considered as a starting point for research rather than an endpoint: For social media data to be integrated with official statistics, we need to develop mechanisms to ensure accuracy. A potential approach could be complementing initial findings with qualitative methods such as structured interviews, conversations with relevant actors, and on-the-ground verification through visits to the communities.
2. Defining geographies and populations are critical to situate insights from social media data: Networked public spheres like Twitter reflect the topics that dominate the local and global discourse, making them an ideal source to grasp trends of public conversation. However, there are two main aspects to consider: 
3. Representation: As we mentioned before, the perspectives and voices we capture in social media are limited and should not be regarded as fully representative of the communities we aim to understand. Specifically, we need to consider the relationship with the technology and internet access of our target communities. Depending on the location of the study there may be limitations of representation in terms of geography, age group, or gender, to name a few.
4. Locality: The scale and granularity of group segmentation cannot always be captured in social media platforms. For example, in our study we decided to focus on gathering data by municipality because we observed that people tended to use hashtags with the name of their municipality rather than their neighborhoods. It is necessary to understand the vocabulary people use to refer to their context; based on that understanding we determine how to map it to the community we aim to examine.  

Local narratives are essential for understanding contexts and perspectives, and opportunities remain to effectively capture and integrate them into development projects. As technology is rapidly adopted into every social interaction, digital traces have become almost ubiquitous, revealing local accounts, habits, patterns, and changes. 

With emerging tools and methodologies, the analysis of social data in large quantities—even in real time and in face of uncertainty—helps us understand a time and place through a multiplicity of perspectives and identify outliers that could inform situated strategies to increase our probability of success. We need, though, to explore further how these approaches can develop further. If you know more examples of how to explore the ways initiatives are connecting with others to extend their impact, or ideas about twitter exploration used for development projects, please share with us at:  acclabmx@undp.org

----------------------------------------

[1] In Mexico, the first confirmed case happened in Mexico City during the last week of February, and the first death from this disease in the country occurred on March 18, 2020.

[2] United Nations Global Pulse. (2013). Mining Indonesian Tweets to Understand the Food Price Crisis. https://www.unglobalpulse.org/project/mining-indonesian-tweets-to-understand-food-price-crises-2013/

[3] United Nations Global Pulse. (2014). Identifying Trends in Discrimination Against Women in the Workplace in Social Media. https://www.unglobalpulse.org/project/feasibility-study-identifying-trends-in-discrimination-against-women-in-the-workplace-in-social-media-2014/

[4] United Nations Global Pulse. (2020). Understanding the COVID-19 Pandemic in Real Time. https://www.unglobalpulse.org/project/understanding-the-covid-19-pandemic-in-real-time/

[5] Stewart, Bonnie. (2016). Twitter as Method: Using Twitter as a Tool to Conduct Research.

6] Clay Alvino. 2021. Estadísticas de la situación digital de México en el 2020-2021. https://branch.com.co/marketingdigital/estadisticas-de-la-situacion-digital-de-mexico-en-el-2020-2021/

[7] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In Workshop Proceedings of the International Conference on Learning Representations (Scottsdale, Arizona, USA) (ICLR ’13).