Closing the artificial intelligence (AI) equity gap includes ensuring AI models and solutions are responsive to local contexts in developing countries. There is an urgent need to reimagine the approach to trust and safety, from reactive and crisis-focused towards proactive, anticipatory and adaptive measures focused on people’s safety and inclusion.

AI is becoming an increasingly powerful technology that can benefit humanity and sustainable development, but empowering local AI ecosystems across all countries needs to be prioritized to build the tools and coalitions necessary to ensure robust safety measures. Here, we make a case for reimagining trust and safety, a critical building block for closing the AI equity gap.
 

How is trust and safety described currently?

"Trust and safety" is not a term that developing countries are always familiar with, yet people are often impacted by it in their everyday interactions. Traditionally, trust and safety refers to the rules and policies that private sector companies put in place to manage and mitigate risks that affect the use of their digital products and services, as well as the operational enforcement systems that determine reactive or proactive restrictions. The decisions that inform these trust and safety practices carry implications for what users can access online, as well as what they can say or do.

While some of these decisions are matters of legal compliance or safety, in other instances, they exist as independent policies that companies create to align with user experience and business interests. Companies tend to independently establish their trust and safety rules with limited reporting, transparency and local collaborations in developing countries.
 

The unintended risk of exclusion

Trust and safety decisions are designed into the fabric of users’ collective digital experiences. Although these filters are often intended as positive and protective measures, without holistic, rights-based considerations, they can become inhibitors that limit people’s ability to fully benefit from digital tools and services.

For instance, trust and safety rules that classify certain regions or user profiles as posing a ‘risk’ can limit their participation in the digital economy, leading to unintended consequences of exclusion. Here are a few scenarios users might encounter:

A ‘hold’ placed on funds
When making or receiving online payments, certain regions or transaction types may be classified as ‘high risk.’ A hold might be placed on the account to safeguard the money, temporarily restricting access to it. Prolonged waiting periods, however, can significantly interrupt daily lives, holding up paying employees or essentials like rent.

Delays to Google Maps listings
In certain regions, businesses attempting to list their company on Google Maps may experience an extensive approval wait time. This is because Google’s trust and safety team cannot easily authenticate new addresses in some countries and cannot easily verify business documents written in some local languages. Not having a business show up on Google search or Maps can impact attracting new business and receiving deliveries.

Blocked from online marketplaces
Vendors of products or services using online marketplaces, such as Airbnb or Alibaba, may be blocked from automated keyword filters, resulting in their listings being blocked. These vendors may often only be permitted to appeal this decision in English or use a less-than-reliable translation software, limiting their ability to contest or resolve the decision if it is a problem of over-enforcement. The inability to do business online can result in a significant loss of income.

While the rationale behind some of these trust and safety practices may be well-intentioned, there are cases where they have led to negative outcomes. Redressal mechanisms that are contextual and localized ought to be considered by companies as an extension of trust and safety implementation. This could mean, for example, using local data to assess and optimize trust and safety to reduce the duration and amount of a hold placed on funds, ensuring an effective feedback loop by integrating local languages.

Nevertheless, the protection and ability to challenge these decisions are not distributed equally. Companies tend to give users in their home markets and major economies greater power and agency in appealing decisions and shaping tech platform policies.

Though companies in developing countries are experiencing accelerated tech adoption, they often face significant barriers to receiving locally applicable interventions that prevent digital harm. This disparity is particularly evident when trust and safety taxonomies and risk frameworks are designed by companies in developed countries without fully accounting for local contexts, cultures and challenges.