Blake builds Claim Framework to analyze and synthesize medical research

Catherine Blake
Catherine Blake, Professor and Associate Dean for Academic Affairs

The news is filled with stories about the latest medical innovations—new prescription drugs that promise relief from symptoms, treatments that offer improved quality of life. Although journalists tend to focus on a single study, people with a chronic health condition who are trying to figure out the best treatment for them, and policy makers who are trying to make recommendations that are best for a community spend much of their time aggregating results from multiple studies.

“There are 24 million abstracts in MEDLINE that capture the best scientific evidence that is available to us. Although anyone can now access this information, rather than clarity, we end up suffering from information overload,” said Associate Professor Catherine Blake. “What we need is a way to analyze and synthesize the results from a collection of articles.”

To ameliorate this problem and work toward improving scientific analysis, Blake has developed the Claim Framework, a rhetorical structure that captures how scientists make claims and a set of tools that use natural language processing to pull out claims made within the journal articles. By attempting to solve the information problem behind this glut of published scientific data, Blake can reveal where there are uncertainties or gaps in the medical literature, as well as provide a detailed analysis of comparative claims.

claimframework.png?itok=IftgDn5h In evidence-based medicine, a detailed analysis of medical research serves as the foundation for decisions about care and evolutions of policy. Working in teams, scientists analyze the multitude of studies and conclusions about a particular drug. Scientists in toxicology use a similar process to conduct risk assessments to understand the risks associated with environmental toxins. Although these groups provide the highest quality evidence, it can take years to conduct a comprehensive review, thus leaving a gap between when a new finding is made and when that finding is factored into public policy.

“We are trying to build a complete picture of what the literature is telling us,” said Blake. “In evidence-based medicine you need to see where there is consensus, but we are also interested in inconsistencies and gaps in the literature. The Claim Framework doesn’t just sweep them under the table, but instead reveals them to us, and these gaps are precisely where we need to focus new research.”

By delving deep into the full text of the article, Blake and her research team capture nuances in the claims that might not be made clear in the abstract. “What is really interesting about this work is that the method does not confuse quantity with quality,” said Blake. “Scientists state their key findings in different ways but just looking for the most frequent word or phrase is not going to give you the key result.”

In a recent study, the research team pulled over 2 million sentences from 9,600 full-text journal articles and then focused on the sentences that talked about Metformin, a drug that is often used to treat diabetes. An automated function pulled out comparisons within the text and then presented a summary of those sentences.

“Only five percent of sentences report a direct comparison, which is a very small percentage, and yet those five percent of sentences contain an abundance of information that captures how effective that drug is both compared with other drugs, and to non-drug interventions and placebos,” said Blake. The resulting analysis conducted by Blake and her team showed what comparisons had not yet been made and thus where new research efforts should be focused.

Not only does this kind of analysis provide details about which specific medical outcomes are improved and under which conditions, it reveals contradictions between study results and shows where new studies should be conducted.

The initial framework was developed as part of a grant Blake received from the National Science Foundation and subsequently published in the Journal of Biomedical Informatics in 2010. In recent years, Blake has continued to explore the framework in different domains, including an early application of the framework to social science that received an honorable mention at the 2013 iConference poster session. Blake presented detailed examples of the framework in medicine, toxicology, and epidemiology at the 2014 Inconsistency Robustness conference held at Stanford University as well as at a meeting of the Research Group for Information Retrieval at the University of Wisconsin, Milwaukee School of Information Studies. It has also been published as a chapter in the book Inconsistency Robustness published by College Publications.

“The beauty of this framework is that it is domain-independent. We have thus far demonstrated the automated methods in medicine, toxicology, and epidemiology, but we are now working on how to apply the framework to other fields where science is informed by empirical evidence,” said Blake.BlakepicClaimFramework.jpg?itok=e_N7wfaz “We need a translational language in order to go from the way the science is written in full text articles to actually summarizing them in a meaningful way. It’s not about the quantity of data that we analyze. It’s about getting at what the result is.”

Ultimately, the Claim Framework will synthesize evidence and highlight new opportunities where further research is necessary to either fill a gap or highlight inconsistent findings.

“It is easy to find your lost keys under a street light, but much harder in the dark. By honing in on the gaps in the scientific literature, on what is missing, we can identify where we need to do more research. And ultimately targeted research will make for better health outcomes for all of us,” said Blake.

Blake's research team from left: master's student Maxwell Isenholt, doctoral student Ana Lucic, Blake, master's student Max McKittrick, and doctoral student Henry Gabb

Tags:
Updated on
Backto the news archive

Related News

Santos honored by Illinois State Comptroller

Professor and Dean Eunice E. Santos was named a 2024 Women's History Month Honoree by the Illinois Office of Comptroller. She was recognized at a ceremony hosted by Illinois Comptroller Susana Mendoza on March 21 in Springfield. At the Women's History Month celebration, Santos and three other women trailblazers were honored for their achievements and contributions to the community.

Eunice Santos

Comics Connection

Associate professor Carol Tilley on Wonder Woman, public libraries vs. drugstores, and our very visual culture.

Carol Tilley in her office surrounded by comics

Book co-edited by Dahlen recognized by ChLA

A book edited by Associate Professor Sarah Park Dahlen and Ebony Elizabeth Thomas, associate professor in the University of Michigan's Marsal Family School of Education, has received the Children's Literature Association (ChLA) Edited Book Award. The annual award recognizes the contributions of an outstanding edited collection of essays to children's literature history, scholarship, and criticism. 

Sarah Park Dahlen

Smith authors paper for newly relaunched ARIST

A paper by Professor Emerita Linda C. Smith, "Reviews and Reviewing: Approaches to Research Synthesis," is one of seven papers included in the relaunch of the Annual Review of Information Science and Technology (ARIST), a collection of peer-reviewed, comprehensive, and systematic reviews on topics relevant to information science.

Linda C. Smith