New study shows LLMs respond differently based on user’s motivation

Chaewon Bak
Michelle Bak
Jessie Chin
Jessie Chin, Assistant Professor

A new study conducted by PhD student Michelle Bak and Assistant Professor Jessie Chin, which was recently published in the Journal of the American Medical Informatics Association (JAMIA), reveals how large language models (LLMs) respond to different motivational states. In their evaluation of three LLM-based generative conversational agents (GAs)—ChatGPT, Google Bard, and Llama 2—the researchers found that while GAs are able to identify users' motivation states and provide relevant information when individuals have established goals, they are less likely to provide guidance when the users are hesitant or ambivalent about changing their behavior.

Bak provides the example of an individual with diabetes who is resistant to changing their sedentary lifestyle.  

"If they were advised by a doctor that exercising would be necessary to manage their diabetes, it would be important to provide information through GAs that helps them increase an awareness about healthy behaviors, become emotionally engaged with the changes, and realize how their unhealthy habits might affect people around them. This kind of information can help them take the next steps toward making positive changes," said Bak.

Current GAs lack specific information about these processes, which puts the individual at a health disadvantage. Conversely, for individuals who are committed to changing their physical activity levels (e.g., have joined personal fitness training to manage chronic depression), GAs are able to provide relevant information and support. 

"This major gap of LLMs in responding to certain states of motivation suggests future directions of LLMs research for health promotion," said Chin.

Bak's research goal is to develop a digital health solution based on using natural language processing and psychological theories to promote preventive health behaviors. She earned her bachelor's degree in sociology from the University of California Los Angeles.

Chin’s research aims to translate social and behavioral sciences theories to design technologies and interactive experiences to promote health communication and behavior across the lifespan. She leads the Adaptive Cognition and Interaction Design (ACTION) Lab at the University of Illinois. Chin holds a BS in psychology from National Taiwan University, an MS in human factors, and a PhD in educational psychology with a focus on cognitive science in teaching and learning from the University of Illinois.

Updated on
Backto the news archive

Related News

Debnath datafies "The Bulletin"

MSIM student Tan Debnath, whose interests span data mining, statistical modeling, text mining, and digital humanities, joined the Center for Children's books as a research assistant. He was tasked with building curation processes that would datafying seventy-five years' worth of archival issues of The Bulletin of the Center for Children's Books, one of the nation's leading children's book review journals.

Tan Debnath stands casually with his hands in his pockets and smiles broadly at the camera. It's a sunny day

He receives Amazon Research Award to improve monitoring of Earth’s ecosystem

A new project led by Professor Jingrui He aims to help scientists monitor disruptions to the Earth’s ecosystem, such as climate change. She recently received support for her work through an Amazon Research Award, which includes $60,000 in cash and an additional $40,000 in Amazon Web Services (AWS) credits.

Jingrui He

iSchool researchers to present at CHI 2025

iSchool faculty and students will present their research at the ACM Conference on Human Factors in Computing Systems (CHI 2025), which will be held from April 26 to May 1 in Yokohama, Japan. 

Undergraduate Research Symposium features iSchool students and mentors

Several iSchool undergraduate students will participate in the 18th annual Undergraduate Research Symposium. During the event, visitors will learn about undergraduate research projects through oral and poster presentations, creative performances, and art exhibits. All are welcome to attend the symposium, which will be held on April 24 from 9:00 a.m.-5:00 p.m. in the Illini Rooms and South Lounge of the Illini Union. Oral presentations will be held on the second floor of the Illini Union.

Wang wins grand prize at Research Live!

Informatics PhD student Olivia Wang won the Grand Prize at the 2025 Research Live! competition, which was held on April 8 in the Campus Instructional Facility Atrium. At the event, which is hosted by the Graduate College, thirteen finalists presented their graduate research in three minutes or less to a general audience. Wang received $500 as the Grand Prize winner.

Olivia Wang