New study examines gender bias in LLMs

Zhixuan Zhou
Kyrie Zhixuan Zhou
Madelyn Sanfilippo
Madelyn Rose Sanfilippo, Assistant Professor

A new study by PhD student Kyrie Zhixuan Zhou and Assistant Professor Madelyn Rose Sanfilippo has uncovered the existence of gender bias in Large Language Models (LLMs) in the U.S. and China. The researchers examined public discussion about this topic on the social media platforms Twitter (now X) and Weibo, finding reports of gender bias in the LLMs ChatGPT and Ernie, China's ChatGPT-equivalent.

Zhou and Sanfilippo were invited to summarize their findings in the Montreal AI Ethics Institute (MAIEI) article, "Public Perceptions of Gender Bias in Large Language Models: Cases of ChatGPT and Ernie." MAIEI is an international nonprofit research institute with a mission to democratize AI ethics literacy. The summary will also be featured in AI Ethics Brief, the institute's weekly newsletter for technical leaders and policymakers around the world.

According to Zhou, finding public discussion on gender bias in ChatGPT on Twitter was straightforward, with over 350 posts on the subject. However, posts about gender bias in Ernie were more difficult to uncover on Weibo, due to China's strict censorship on online discussions about feminism. Zhou and Sanfilippo found that "cultural factors impact biases in LLMs since they were trained on data that reflect those cultural contexts." They observed implicit gender bias in discussion about ChatGPT, e.g., associating different genders with different professional titles, but explicit gender bias in discussion about Ernie, e.g., devaluing women by their age.

"Gender bias is always a severe and under-investigated issue in information technology," said Zhou. "By understanding and proposing ways to solve gender bias in AI, we aim to create a more inclusive and fair tech infrastructure in the LLM era."

To rectify the situation, the researchers propose governance recommendations to regulate gender bias in LLMs as well as the development of educational tutorials to equip AI creators with awareness and knowledge of AI gender bias.

Zhou's research interests are in tech ethics and accessibility. He aims to understand, design, and govern ICT/AI experience for vulnerable populations such as older adults, people with disabilities, women in conservative cultures, and people in authoritarian regimes. Zhou earned his BEng in computer science from Wuhan University.

Sanfilippo uses mixed methods to address research questions about participation in and the legitimacy of sociotechnical governance; social justice issues associated with sociotechnical governance; privacy in sociotechnical systems; and differences between policies or regulations and sociotechnical practice. Her work practically supports decision-making in, management of, and participation in a diverse public sphere. She earned her MS and PhD in information science from Indiana University.

Updated on
Backto the news archive

Related News

Hoiem receives Schiller Prize for “Education of Things”

Associate Professor Elizabeth Hoiem has won the 2025 Justin G. Schiller Prize from The Bibliographical Society of America for her book, The Education of Things: Mechanical Literacy in British Children's Literature, 1762-1860 (University of Massachusetts Press). The prize, which recognizes the best bibliographical work on pre-1951 children's literature, includes a cash award of $3,000 and a year's membership in the Society. 

Elizabeth Hoiem

Chan authors new book connecting eugenics and Big Tech

Associate Professor Anita Say Chan has authored a new book that identifies how the eugenics movement foreshadows the predatory data tactics used in today's tech industry. Her book, Predatory Data: Eugenics in Big Tech and Our Fight for an Independent Future, was released this month by the University of California Press and featured in the news outlets San Francisco Chronicle and Mother Jones.

Anita Say Chan

CCB contributes to new Books to Parks site on Lyddie

The Center for Children's Books (CCB) collaborated with the National Park Service (NPS) to launch a new Books to Parks website on Lyddie, a 1991 novel by Katherine Paterson that highlights the experiences of young women working in textile mills in nineteenth-century Lowell, Massachusetts. 

Lyddie book

Layne-Worthey edits book on digital humanities and LIS

Glen Layne-Worthey, associate director for research support services for the HathiTrust Research Center (HTRC), and Isabel Galina, researcher at the Institute for Bibliographic Studies at the National University of Mexico, have edited a new book, The Routledge Companion to Libraries, Archives, and the Digital Humanities, which was recently released by Routledge.

Glen Layne-Worthey

Wang group to present at BigData 2024

Members of Associate Professor Dong Wang's research group, the Social Sensing and Intelligence Lab, will present their research at the 2024 IEEE International Conference on Big Data (BigData 2024), which will be held from December 15-18 in Washington, D.C. BigData 2024 is the premier venue to present and discuss progress in research, development, standards, and applications of topics in artificial intelligence, machine learning and big data analytics.

Dong Wang