Introduction to Nitesh Shawla and His Concerns
Nitesh Shawla, director of the Lucy Family Institute for Data and Society at the University of Notre Dame, has raised concerns about artificial intelligence (AI) models such as ChatGPT perpetuating cultural, linguistic, and social biases. These models, including Gemini and Llama, are transforming how we interact with technology but also propagate prejudices due to their training on dominant Anglo-Saxon and male-centric datasets.
The Challenge of Responsible AI
Shawla highlighted the difficulty in defining responsible AI, as it lacks a clear understanding of what constitutes responsibility. Without this clarity, he questioned how algorithms can optimize for it. During the opening speech at the XVII Mexican Congress on Artificial Intelligence, Shawla pointed out that while AI has advanced through mathematical optimizations like regression, neural networks, or loss functions, responsibility cannot be easily encoded into an objective function.
Lack of Equity and Cultural Inclusion Optimization
Shawla’s lab has proposed solutions like Fair Mixture of Experts (Fair MOE), an AI architecture that combines experts specialized in accuracy with others focused on interpretability. This approach allows users to allocate an “interpretability budget,” choosing between accurate or comprehensible models. This flexibility would enable the use of AI in regulated domains, such as banking loans, without sacrificing transparency.
Cultural and Linguistic Biases
One of the most critical aspects of Shawla’s talk was the denunciation of linguistic and cultural bias in large language models (LLMs). These models have been primarily trained on publicly available English data, not just the language but also the cultural norms it carries.
Cultural Sensitivity vs. Linguistic Comprehension
Shawla explained that the issue goes beyond literal translation. He cited an example of a Kenyan woman using a chatbot to express, in her language, that “the baby is not at risk,” an idiomatic expression for complications during pregnancy. The model, trained without cultural context, suggested toys instead.
Challenges in Latin America
In Latin America, where Spanish variants vary significantly between countries and regions, the challenges are even greater. Shawla warned that what is said in Mexico might sound incomprehensible or even offensive elsewhere. He suggested that each region should develop its own language models integrating both language and cultural context.
Building Culturally Localized Language Models
Shawla is currently working with several Latin American countries, including Chile, to build culturally localized language models. He emphasized that simply translating ChatGPT is insufficient; instead, models must understand the realities of each community.
Gender Bias in AI Models
Shawla’s team conducted an experiment demonstrating how models like ChatGPT can reinforce gender biases. By asking the model to predict whether a person earns more than $50,000 annually with identical profiles except for gender, the model responded that the man probably did while the woman likely did not. When asked why, the system referenced statistical patterns without questioning its assumptions.
Historical Data Reflection
Shawla explained that these biases are not accidental but reflect the historical data used to train the models. Early large-scale internet users were predominantly men in Western contexts, which is reflected in the training datasets.
Responsible AI Principles
To tackle these challenges, Shawla proposes a bioethics-inspired approach centered on two beneficence principles: avoiding harm and maximizing well-being. However, he acknowledged that these principles often conflict.
Unforeseen Consequences in AI
Shawla explained that unlike medications with known side effects, AI systems lack transparency regarding potential consequences. He warned against repeating the mistake made with social media, where their consequences were not foreseen when they were first introduced.
Concrete Projects Applying Responsible AI
Colombian Truth Commission: Shawla’s team collaborated with the Colombian Truth Commission to develop a model trained exclusively on Colombian testimonies, resulting in much more specific responses than models like GPT-4.
Hospital Infantil de México:
In Mexico’s Hospital Infantil de México Federico Gómez, Shawla and his team are helping digitize clinical records. They’ve implemented a system that scans, structures, and creates integrated electronic records with social data, including nutrition, housing, and mental health. This information will suggest fairer and more effective treatments.
Key Questions and Answers
- What are the concerns raised by Nitesh Shawla regarding AI models? Shawla is concerned that AI models like ChatGPT perpetuate cultural, linguistic, and social biases due to their training on dominant Anglo-Saxon and male-centric datasets.
- How does Shawla propose to address these challenges? He suggests a bioethics-inspired approach centered on beneficence principles, acknowledging the conflict between avoiding harm and maximizing well-being. He also emphasizes the need for transparency in AI systems regarding potential consequences.
- What examples does Shawla provide to illustrate AI biases? Shawla demonstrates how AI models can reinforce gender biases and lack cultural sensitivity, using examples from chatbots misinterpreting idiomatic expressions.
- How are Shawla’s principles of responsible AI being applied in real-world projects? Shawla shares two concrete projects: a model for the Colombian Truth Commission providing specific responses based on Colombian testimonies and a digital system at Mexico’s Hospital Infantil de México Federico Gómez integrating clinical data with social determinants for better treatment suggestions.
