iSchool Assistant Professor JooYoung Seo is developing a data visualization tool that will help make visual representations of statistical data accessible to researchers and students who are blind or visually impaired.
The multimodal representation tool is aimed at the accessibility of statistical graphs, such as bar plots, box plots, scatter plots and heat maps.
"Sighted people can pick up a great deal of insight and get the big picture from visualization, but visualized data is very challenging to those who are visually impaired," said Seo, whose research includes accessible computing, universal design and inclusive data science. Seo, who is blind, is a certified accessibility expert. He also is affiliated with U. of I.'s National Center for Supercomputing Applications, where he is addressing accessibility issues for a National Science Foundation-funded high-performance computing project.
The tool, called Multimodal Access and Interactive Data Representation, presents data through sonification, text and Braille. For example, sonification can use different tones for the information on a bar plot. Seo uses stereo sound that is mapped to represent the X and Y axes. Certain tones represent data from left to right on the X axis, and higher and lower pitches represent the Y axis values—the higher the Y value, the higher the pitch. Similarly, the smoothed line of a scatter plot is presented aurally, and users also can navigate point by point using sound and Braille.
The tool uses a refreshable Braille display, which converts words on a screen – or in the case of Seo's tool, representations of data – to a tactile pattern on a Braille display. Users of the system who are blind or have low vision can "quickly understand the visualization in an alternative fashion," Seo said. The tool also converts the data to Braille characters on the screen.
"I really want to push the boundaries of current technology. Accessibility is limited to what technology can offer. If our technology can provide multimodalities, it simultaneously can provide richer information," he said.
Seo recently received a $650,000 grant from the Institute of Museum and Library Services that will help him make the tool more broadly available. He also was one of 19 researchers nationwide that were awarded grants from Teach Access to make their college courses more accessible. He's creating a module for his data science course on making data visualizations and data workflows accessible.
His accessibility module will be added to the Teach Access repository, and Seo plans to share it on GitHub as an open-source project. He'll also introduce it to his data science students during this academic year.
"I wanted to teach students who don't necessarily have any disabilities about other types of data representations such as sonification and verbal identification, so when they graduate and go into industry or academia as data scientists, they can produce more accessible data visualization for all," Seo said.
"Data science itself is accessible, but the problem comes from the final representation – the visualized data," he said. "The essence of STEM is not visual, but when people learn STEM-related content, the subjects are full of visual materials. It's why blind people are marginalized. Once we have access to non-visual or alternative methods, we can access the essence of science, which is just concepts."
While accessibility issues arise from the methods by which data is represented, those looking at the data also need data literacy skills to interpret it, Seo said.
"It’s really important for blind and visually impaired people to have strong data literacy. If we are given any raw data or reproducible code, we have to interpret how the data was processed. Blind and visually impaired people are very marginalized in the data literacy movement," he said. "I'm really trying to provide more opportunities for other blind folks to improve their data literacy."
In August, Seo led a two-week online data science course, focused on data visualization, for blind participants in collaboration with the Smith-Kettlewell Eye Research Institute, funded by the National Institute on Disability, Independent Living and Rehabilitation Research.