Haocong Cheng's Dissertation Defense
PhD candidate Haocong Cheng will present his dissertation defense, “Enhancing Accessible Video-based learning for Deaf and Hard-of-Hearing Learners with AI Technologies.” His dissertation committee includes Professor Stephen Downie (Chair); Associate Professor Rachel Adler, Director of Research; Associate Professor Nigel Bosch; and Professor Qi Wang, Gallaudet University.
Abstract
D/deaf and Hard-of-Hearing (DHH) learners face unique challenges in video-based learning due to the complex interplay between visual and auditory information in videos. Conventional accessibility research on video-based learning has mainly focused on captions and interpreters, which may not sufficiently address the accessibility requirements for delivering multi-modal content in educational videos. With the growing capabilities in Artificial Intelligence (AI) technologies, such as Large Language Models (LLMs), new opportunities exist to provide personalized learning to support DHH learners and educators, but little is known about how such technologies can be optimally designed and applied to improve accessibility in video-based learning.
This dissertation explores the use of AI technologies to meet the accessibility needs of DHH learners in video-based learning. First, it investigates how AI can support DHH learners directly. Through studies of emotional experiences during video-based learning and interactions with LLM-powered AI tutors, the findings reveal differences in how DHH learners engage with mainstream educational videos and show that AI tutors with DHH education experience are perceived as more human-like and trustworthy because of their cultural knowledge of DHH communities. Second, this dissertation examines how educational videos can be redesigned to better suit the preferences of DHH learners. Through formative and evaluative studies with DHH learners and educators with experience teaching DHH learners, a set of motion-driven design ideas was developed to address challenges related to visual-audio relevance, visual attention switching, information density, and pacing. The results show that improving visual-audio relevance and guiding visual attention can enhance learning experiences and reduce cognitive demands. Third, this dissertation investigates how AI technologies can help educators create accessible educational videos. Studies conducted with mainstream educators, DHH education experts, and DHH learners reveal both the opportunities and limitations of current AI-powered video generation tools, highlighting the continued need for human oversight to ensure accessibility and pedagogical quality.
Together, these findings contribute new knowledge on DHH learners’ experiences in video-based learning, provide design guidance to create more accessible educational videos, and identify opportunities to develop human-centered AI technologies that support accessibility in education.
Questions? Contact Haocong Cheng.