Willis defends dissertation

Craig Willis
Craig Willis, Teaching Assistant Professor

Doctoral candidate Craig Willis successfully defended his dissertation, "Trust, But Verify: An Investigation of Methods of Verification and Dissemination of Computational Research Artifacts for Transparency and Reproducibility," on June 15.

His committee included Associate Professor Victoria Stodden (chair); Assistant Professor Peter Darch; Professor Bertram Ludäscher; and Michela Taufer, professor of electrical engineering and computer science at the University of Tennessee, Knoxville.

From the abstract: In this study, I investigate how research communities are addressing concerns about the quality and rigor of computational research. I focus specifically on initiatives to expand the peer review and publication process to include new requirements for the assessment and dissemination of computational research artifacts. I report the results of a multiple-case analysis of two primary (American Journal of Political Science, ACM/IEEE Supercomputing) and five supplemental cases in political science, computer science, economics, mathematics, and statistics. Cases were developed through qualitative analysis based on interviews with key stakeholders (n= 17) including editors, reviewers, and verifiers; a sample of (n= 27) verified artifacts; and documentary evidence including policies, guidelines, and workflows. Based on the cases, I identify key factors that influence the operationalization of policies and workflows; the elements that each community considers important to the assessment of computational transparency and reproducibility; as well as the tools and infrastructure that they leverage to aid in the creation, assessment and dissemination of reproducible research artifacts. I develop a framework to analyze the reproducibility initiatives and a conceptual model of reproducible research artifacts. I relate my findings to recommendations from the recent National Academies of Sciences, Engineering and Medicine (NASEM) report on Reproducibility and Replicability in Science and provide a set of normative guidelines for communities interested in pursuing similar initiatives with implications for journal and conference leadership; tool and infrastructure developers; and funding bodies. I conclude that, while promising, further efforts should be made to increase our understanding of the effect of initiative policies and technological advancements on research quality.

Research Areas:
Tags:
Updated on
Backto the news archive

Related News

Get to know Cadence Cordell, MSLIS student

Cadence Cordell was inspired by her undergraduate work experience to pursue a degree in library and information science. She followed in her mother’s footsteps by selecting the iSchool for her MSLIS. After completing a recent research poster presentation, she combined her scholarly pursuit with her hobby by sewing her fabric poster into a squirrel plushie.

Cadence Cordell

Recent graduate committed to making libraries accessible and inclusive

Joshua Short knows firsthand the barriers to public library access that patrons living on modest wages experience. Having grown up in a self-professed "low-income environment," Short has made it his mission to reduce these barriers, such as library fines, inadequate transportation, and limited computer literacy.

Joshua Short

Spectrum Scholar Spotlight: Leslie Lopez

Twelve iSchool master's students were named 2024–2025 Spectrum Scholars by the American Library Association (ALA) Office for Diversity, Literacy, and Outreach Services. This “Spectrum Scholar Spotlight” series highlights the School’s scholars. MSLIS student Leslie Lopez graduated from the University of North Texas with a BA in psychology.

Leslie Lopez headshot

SafeRBot to assist community, police in crime reporting

Across the nation, 911 dispatch centers are facing a worker shortage. Unfortunately, this understaffing, plus the nature of the job itself, leads to dispatchers who are often overworked and stressed. Meanwhile, when community members need to report a crime, their options are to contact 911 for an emergency or, in a non-emergency situation, call a non-emergency number or fill out an online form. A new chatbot, SafeRBot, designed and developed by Associate Professor Yun Huang, Informatics PhD student Yiren Liu, and BSIS student Tony An seeks to improve the reporting process for non-emergency situations for both community members and dispatch centers.

Yun Huang

New digital collection sheds light on queer nightlife in Champaign County

Adam Beaty decided to pursue an MSLIS degree to combine his love of history, the arts, and community-centered spaces. This combination of interests culminated in a 244-item digital collection that showcases digitized materials depicting nearly thirty years of queer nightlife in Champaign County. 

Adam Beaty_headshot