Youth-AI-Safety named a winning team in international hackathon

Yaman Yu
Yaman Yu
Yang Wang
Yang Wang, Professor
Yun Huang
Yun Huang, Associate Professor

A team of researchers from the SALT (Social Computing Systems) Lab has been selected as a winner in an international hackathon hosted by the Berkeley Center for Responsible, Decentralized Intelligence. The LLM Agents MOOC Hackathon brought together over 3,000 students, researchers, and practitioners from 127 countries to build and showcase innovative work in large language model (LLM) agents, grow the AI agent community, and advance LLM agent technology.

Hackathon teams could compete in one of five tracks: Applications, Benchmarks, Fundamentals, Safety, and Decentralized and Multi-Agents. Team Youth-AI-Safety, which included PhD student Yaman Yu, Informatics PhD student Yiren Liu, Computer Science undergraduate Jacky Zhang, Associate Professor Yun Huang, and Professor Yang Wang, won 8th place in the Applications Track—the most popular track with 75 percent of the participants.

The team's hackathon project builds on members' earlier work highlighting the risks teenagers face when using generative artificial intelligence (GAI)—risks that parents are often unaware of and struggling to manage, Yu said. The goal of the project is to safeguard youth in GAI platforms by building a risk taxonomy for youth-nuanced challenges; creating a real-world labeled dataset for risk detection; and developing YouthSafeAgent, a parental control system for real-time monitoring and family communication.

“We collected chat logs from youth participants, a large-scale Reddit dataset, and an AI incident database to categorize risks and develop a structured taxonomy for youth-GAI interactions. Using this taxonomy, we labeled the chat logs to create a dataset. Based on this, we developed a parental control tool that enables real-time monitoring of children's risky interactions with GAI and allows parents to take action when needed,” Yu explained.

Future steps for the project include expanding the dataset, adding real-time intervention strategies, integrating user feedback from parents and youth, and finally releasing the YouthSafeAgent risk taxonomy, dataset, and parental control tools.

The SALT Lab seeks to understand how people interact with sociotechnical computing systems and develops original designs that either enable new forms of user/social interaction or impact existing interaction. The lab proposes, develops, and evaluates novel infrastructure solutions for a variety of social computing applications, including mobile crowdsourcing systems for public safety, systems for protecting people's privacy in social media, designs for enabling new forms of interactions between librarians and patrons, and mechanisms for making authentication more accessible for people with disabilities.

Updated on
Backto the news archive

Related News

Faculty receive support for AI-related projects from new pilot program

Associate Professor Yun Huang, Assistant Professor Jiaqi Ma, and Assistant Professor Haohan Wang have received computing resources from the National Artificial Intelligence Research Resource (NAIRR), a two-year pilot program led by the National Science Foundation in partnership with other federal agencies and nongovernmental partners. The goal of the pilot is to support AI-related research with particular emphasis on societal challenges. Last month, awardees presented their research at the NAIRR Pilot Annual Meeting.

Winning exhibits highlight evolution of music media and Uni High magazine

MSLIS students Monica Gil, Holly Bleeden, and Harrison Price were selected as winners of this year's Graduate Student Exhibit Contest, sponsored by the University of Illinois Library. Gil and Bleeden won first place for their exhibit, "Echoes of Time: The Evolution of Music Media," and Price won second place for his exhibit, "Unique-ly Illinois: Creative Writing from High School to Higher Education." The exhibits will be on display in the Marshall Gallery in the library through the end of March.

MSLIS students Monica Gil and Holly Bleeden standing next to their exhibit, "Echoes of Time: The Evolution of Music Media," at the Main Library.

Wei receives Amazon Post Internship Fellowship

PhD student Tianxin Wei has been awarded an Amazon Post Internship Fellowship, which will provide $20,000 in unrestricted funds and $20,000 in Amazon Web Services (AWS) credits to support Wei's research with his advisor, Professor Jingrui He. For the past two summers, Wei has served as an applied scientist intern at Amazon in Palo Alto, California. He has been part of a team that is working on search query understanding within Amazon apps and services, as well as developing shopping foundation models.

Tianxin Wei

iSchool participation in iConference 2025

The following iSchool faculty and students will participate in iConference 2025, which will be held virtually from March 11-14 and physically from March 18-22 in Bloomington, Indiana. The theme of this year's conference is "Living in an AI-gorithmic world."

Carboni joins the iSchool faculty

The iSchool is pleased to announce that Nicola Carboni has joined the faculty as an assistant professor. He previously served as a postdoctoral researcher and lecturer in digital humanities at the University of Geneva.

Nicola Carboni