In a study released yesterday, OpenAI has shed light on the fairness of its popular language model, ChatGPT. The research, which analysed millions of real user interactions, found that the AI chatbot exhibits minimal bias when responding to users with names associated with different genders, races, or ethnicities.
OpenAI's approach to this study prioritised user privacy while examining real-world usage patterns. Researchers employed a "Language Model Research Assistant" (LMRA), powered by GPT-4o, to analyse trends across a vast number of ChatGPT transcripts without accessing the underlying conversations.
The study's findings are encouraging for those concerned about AI bias. ChatGPT consistently provided high-quality answers regardless of the gender or racial connotations of the user's name. In fact, less than 0.1% of overall cases showed differences in responses that reflected harmful stereotypes based on a name's association with gender, race, or ethnicity.
However, the research did uncover some areas for improvement. In certain domains, older models showed biases up to around 1%. Notably, the GPT-3.5 Turbo model demonstrated the highest level of bias, while newer models all had less than 1% bias across all tasks.
The study examined responses across various domains, including art, business & marketing, education, employment, entertainment, health-related topics, legal matters, technology, and travel. Interestingly, open-ended tasks with longer responses, such as "Write a story," were more likely to include harmful stereotypes.
To ensure the accuracy of the LMRA's assessments, both the language model and human raters evaluated the same public chats. For gender-related stereotypes, there was a high level of agreement, with the language model's answers aligning with human raters' answers more than 90% of the time. However, agreement rates were lower for racial and ethnic stereotypes, highlighting an area for further research.
OpenAI acknowledges several limitations of their study. The research primarily focused on English-language interactions and considered binary gender associations based on common U.S. names. It also covered only four races and ethnicities (Black, Asian, Hispanic, and White) and was limited to text interactions. The company plans to build on this research to improve fairness more broadly.
OpenAI has incorporated this methodology into its standard suite of model performance evaluations. Furthermore, the company has shared detailed system messages to enable external researchers to conduct their own fairness experiments.