Understanding and Addressing Tribal Identity Biases in AI Systems
Artificial intelligence (AI) systems, such as large language models (LLMs), are increasingly being integrated into our daily lives. However, recent research has shown that these systems can develop biases similar to those of humans, favoring their “in-group” and disparaging “out-groups.” This phenomenon is known as social identity bias.
The Prevalence of Social Identity Bias in AI Systems
A study published in the journal Nature Computational Science found that dozens of LLMs, including baseline models and more advanced ones, exhibited social identity biases. The researchers generated 2,000 sentences with indications of “us versus them” dynamics and let the models complete the sentences. The results showed that “We are” statements were more positive, while “They are” statements were more negative.
Modifying Social Identity Biases in AI Systems
The researchers also sought to determine whether these biases could be modified by changing the way LLMs were trained. They found that fine-tuning the LLM with partisan social media data increased intergroup solidarity and hostility toward outgroups. However, when they filtered out phrases expressing favoritism toward one group and hostility toward another group from the same social media data, they were able to effectively reduce these polarizing effects.
Implications for AI Development and Training
The study’s findings suggest that AI biases can be reduced by carefully selecting the data used to train these systems. This has significant implications for AI development and training, as it highlights the importance of considering the potential biases in the data used to train AI systems.
Image: DALL-E