Youth-AI-Safety named a winning team in international hackathon

Yaman Yu
Yaman Yu
Yang Wang
Yang Wang, Professor
Yun Huang
Yun Huang, Associate Professor

A team of researchers from the SALT (Social Computing Systems) Lab has been selected as a winner in an international hackathon hosted by the Berkeley Center for Responsible, Decentralized Intelligence. The LLM Agents MOOC Hackathon brought together over 3,000 students, researchers, and practitioners from 127 countries to build and showcase innovative work in large language model (LLM) agents, grow the AI agent community, and advance LLM agent technology.

Hackathon teams could compete in one of five tracks: Applications, Benchmarks, Fundamentals, Safety, and Decentralized and Multi-Agents. Team Youth-AI-Safety, which included PhD student Yaman Yu, Informatics PhD student Yiren Liu, Computer Science undergraduate Jacky Zhang, Associate Professor Yun Huang, and Professor Yang Wang, won 8th place in the Applications Track—the most popular track with 75 percent of the participants.

The team's hackathon project builds on members' earlier work highlighting the risks teenagers face when using generative artificial intelligence (GAI)—risks that parents are often unaware of and struggling to manage, Yu said. The goal of the project is to safeguard youth in GAI platforms by building a risk taxonomy for youth-nuanced challenges; creating a real-world labeled dataset for risk detection; and developing YouthSafeAgent, a parental control system for real-time monitoring and family communication.

“We collected chat logs from youth participants, a large-scale Reddit dataset, and an AI incident database to categorize risks and develop a structured taxonomy for youth-GAI interactions. Using this taxonomy, we labeled the chat logs to create a dataset. Based on this, we developed a parental control tool that enables real-time monitoring of children's risky interactions with GAI and allows parents to take action when needed,” Yu explained.

Future steps for the project include expanding the dataset, adding real-time intervention strategies, integrating user feedback from parents and youth, and finally releasing the YouthSafeAgent risk taxonomy, dataset, and parental control tools.

The SALT Lab seeks to understand how people interact with sociotechnical computing systems and develops original designs that either enable new forms of user/social interaction or impact existing interaction. The lab proposes, develops, and evaluates novel infrastructure solutions for a variety of social computing applications, including mobile crowdsourcing systems for public safety, systems for protecting people's privacy in social media, designs for enabling new forms of interactions between librarians and patrons, and mechanisms for making authentication more accessible for people with disabilities.

Updated on
Backto the news archive

Related News

Zhou defends dissertation

Doctoral candidate Kyrie Zhixuan Zhou successfully defended his dissertation, "A Pragmatic and Human-centered Approach to Promoting Software Accessibility: Design, Education, Governance," on April 3.

Zhixuan Zhou

Knox appointed interim dean

Professor Emily Knox has been appointed to serve as interim dean of the School of Information Sciences, pending approval by the Board of Trustees. Until officially approved, her title will be interim dean designate. The appointment will begin April 1, 2025.

Emily Knox

iSchool instructors ranked as excellent

Fifty-six iSchool instructors were named in the University's List of Teachers Ranked as Excellent for Fall 2024 and Winter 2024-2025. The rankings are released every semester, and results are based on the ratings from the Instructor and Course Evaluation System (ICES) questionnaire forms maintained by Measurement and Evaluation in the Center for Innovation in Teaching and Learning. 

iSchool Building

Scholarship alleviates financial burden for returning student

During her time as an active-duty Naval Officer, Anna Hartman realized that she had a passion for helping others and building community. That passion, combined with a lifelong love of reading, led her to pursue an MSLIS degree at the University of Illinois. Hartman is receiving support for her studies through the Balz Endowment Fund, which was established by Nancy (BA LAS '70, MSLIS '72) and Dan (BS Media '68, MS Media '72) Balz to help make education more affordable for returning students.

Anna Hartman

Ocepek and Sanfilippo co-edit book on misinformation

Assistant Professor Melissa Ocepek and Assistant Professor Madelyn Rose Sanfilippo have co-edited a new book, Governing Misinformation in Everyday Knowledge Commons, which was recently published by Cambridge University Press. An open access edition of the book is available, thanks to support from the Governing Knowledge Commons Research Coordination Network (NSF 2017495). The new book explores the socio-technical realities of misinformation in a variety of online and offline everyday environments. 

Governing Misinformation in Everyday Knowledge Commons book