Willis defends dissertation

Craig Willis
Craig Willis

Doctoral candidate Craig Willis successfully defended his dissertation, "Trust, But Verify: An Investigation of Methods of Verification and Dissemination of Computational Research Artifacts for Transparency and Reproducibility," on June 15.

His committee included Associate Professor Victoria Stodden (chair); Assistant Professor Peter Darch; Professor Bertram Ludäscher; and Michela Taufer, professor of electrical engineering and computer science at the University of Tennessee, Knoxville.

From the abstract: In this study, I investigate how research communities are addressing concerns about the quality and rigor of computational research. I focus specifically on initiatives to expand the peer review and publication process to include new requirements for the assessment and dissemination of computational research artifacts. I report the results of a multiple-case analysis of two primary (American Journal of Political Science, ACM/IEEE Supercomputing) and five supplemental cases in political science, computer science, economics, mathematics, and statistics. Cases were developed through qualitative analysis based on interviews with key stakeholders (n= 17) including editors, reviewers, and verifiers; a sample of (n= 27) verified artifacts; and documentary evidence including policies, guidelines, and workflows. Based on the cases, I identify key factors that influence the operationalization of policies and workflows; the elements that each community considers important to the assessment of computational transparency and reproducibility; as well as the tools and infrastructure that they leverage to aid in the creation, assessment and dissemination of reproducible research artifacts. I develop a framework to analyze the reproducibility initiatives and a conceptual model of reproducible research artifacts. I relate my findings to recommendations from the recent National Academies of Science, Engineering and Medicine (NASEM) report on Reproducibility and Replicability in Science and provide a set of normative guidelines for communities interested in pursuing similar initiatives with implications for journal and conference leadership; tool and infrastructure developers; and funding bodies. I conclude that, while promising, further efforts should be made to increase our understanding of the effect of initiative policies and technological advancements on research quality.

Research Areas:
Tags:
Updated on
Backto the news archive

Related News

iSchool students, recent graduate selected for Library of Congress Junior Fellows program

MS students Katherine Howell and Randi Proescholdt and recent graduate Patty Templeton (MS '20) are spending this summer as Library of Congress Junior Fellows. This competitive internship program, which is being held remotely for the first time, provides students with the opportunity to explore digital initiatives and increase access to the institution’s collections while working under the direction of library curators and specialists in various divisions.

Grace receives NASIG scholarship

MS student Madina Grace is the recipient of a Fritz Schwartz Education Scholarship from NASIG, an independent organization working to advance and transform the management of information resources. The award provides enhanced educational opportunities to an aspiring library student who has prior experience with managing information resources.

Madina Grace

Turner receives CARLI Scholarship

Master's student Victoria Turner has been selected to receive a scholarship from the Consortium of Academic and Research Libraries in Illinois (CARLI). Awarded for the first time in the 2020-2021 academic year, the CARLI Scholarship provides financial assistance to current employees of CARLI Governing Member Libraries pursuing graduate studies leading to a master's degree in library and information science at the University of Illinois. CARLI provides library services and support to 128 Illinois public universities, community colleges, private colleges and universities, and special libraries.

Victoria Turner