Willis defends dissertation

Craig Willis
Craig Willis

Doctoral candidate Craig Willis successfully defended his dissertation, "Trust, But Verify: An Investigation of Methods of Verification and Dissemination of Computational Research Artifacts for Transparency and Reproducibility," on June 15.

His committee included Associate Professor Victoria Stodden (chair); Assistant Professor Peter Darch; Professor Bertram Ludäscher; and Michela Taufer, professor of electrical engineering and computer science at the University of Tennessee, Knoxville.

From the abstract: In this study, I investigate how research communities are addressing concerns about the quality and rigor of computational research. I focus specifically on initiatives to expand the peer review and publication process to include new requirements for the assessment and dissemination of computational research artifacts. I report the results of a multiple-case analysis of two primary (American Journal of Political Science, ACM/IEEE Supercomputing) and five supplemental cases in political science, computer science, economics, mathematics, and statistics. Cases were developed through qualitative analysis based on interviews with key stakeholders (n= 17) including editors, reviewers, and verifiers; a sample of (n= 27) verified artifacts; and documentary evidence including policies, guidelines, and workflows. Based on the cases, I identify key factors that influence the operationalization of policies and workflows; the elements that each community considers important to the assessment of computational transparency and reproducibility; as well as the tools and infrastructure that they leverage to aid in the creation, assessment and dissemination of reproducible research artifacts. I develop a framework to analyze the reproducibility initiatives and a conceptual model of reproducible research artifacts. I relate my findings to recommendations from the recent National Academies of Sciences, Engineering and Medicine (NASEM) report on Reproducibility and Replicability in Science and provide a set of normative guidelines for communities interested in pursuing similar initiatives with implications for journal and conference leadership; tool and infrastructure developers; and funding bodies. I conclude that, while promising, further efforts should be made to increase our understanding of the effect of initiative policies and technological advancements on research quality.

Research Areas:
Updated on
Backto the news archive

Related News

Cheng defends dissertation

Doctoral candidate Jessica Cheng successfully defended her dissertation, "Agreeing to Disagree: Applying a Logic-based Approach to Reconciling and Merging Multiple Taxonomies," on May 25. 

Jessica Cheng

Student award recipients announced

Each year, the School of Information Sciences recognizes a group of outstanding students for their achievement in academics as well as a number of attributes that contribute to professional success. Congratulations to this year's honorees!

Alma with cap

iSchool students present their research at Urbana City Council meeting

At the Urbana City Council meeting on May 9, students in the Community Data (IS 594) course presented their research on how communities are reducing gun violence. According to their instructor Chamee Yang, postdoctoral research associate with the iSchool, Community Data Clinic, and Just Infrastructures Initiative, the new course was designed as an experiential learning opportunity with a community engagement component, where students could gain research experience with real-world implications. Throughout the Spring 2022 semester, students worked in groups to explore community-driven approaches to prevent gun violence.

Chamee Yang, Sarah Unruh, and Gowri Balasubramaniam

Dinh defends dissertation

Doctoral candidate Ly Dinh successfully defended her dissertation, "Advances to Network Analysis Theories and Methods for the Understanding of Formal and Emergent Structures in Interpersonal, Corporate/Organizational, and Hazards Response Setting," on May 19.

Ly Dinh

Lee selected for leadership institute

MS/LIS student Kyra Lee had the opportunity to network with leaders in the LIS field at the 2022 Black Caucus American Library Association (BCALA) Leadership Institute. At the inaugural event, which took place from April 12-14 in Durham, North Carolina, LIS students and early career library professionals gathered for workshops, panels, facilitated discussions, and presentations. Lee was one of eighteen students selected to participate in the institute.

Kyra Lee