Willis defends dissertation

Craig Willis
Craig Willis

Doctoral candidate Craig Willis successfully defended his dissertation, "Trust, But Verify: An Investigation of Methods of Verification and Dissemination of Computational Research Artifacts for Transparency and Reproducibility," on June 15.

His committee included Associate Professor Victoria Stodden (chair); Assistant Professor Peter Darch; Professor Bertram Ludäscher; and Michela Taufer, professor of electrical engineering and computer science at the University of Tennessee, Knoxville.

From the abstract: In this study, I investigate how research communities are addressing concerns about the quality and rigor of computational research. I focus specifically on initiatives to expand the peer review and publication process to include new requirements for the assessment and dissemination of computational research artifacts. I report the results of a multiple-case analysis of two primary (American Journal of Political Science, ACM/IEEE Supercomputing) and five supplemental cases in political science, computer science, economics, mathematics, and statistics. Cases were developed through qualitative analysis based on interviews with key stakeholders (n= 17) including editors, reviewers, and verifiers; a sample of (n= 27) verified artifacts; and documentary evidence including policies, guidelines, and workflows. Based on the cases, I identify key factors that influence the operationalization of policies and workflows; the elements that each community considers important to the assessment of computational transparency and reproducibility; as well as the tools and infrastructure that they leverage to aid in the creation, assessment and dissemination of reproducible research artifacts. I develop a framework to analyze the reproducibility initiatives and a conceptual model of reproducible research artifacts. I relate my findings to recommendations from the recent National Academies of Sciences, Engineering and Medicine (NASEM) report on Reproducibility and Replicability in Science and provide a set of normative guidelines for communities interested in pursuing similar initiatives with implications for journal and conference leadership; tool and infrastructure developers; and funding bodies. I conclude that, while promising, further efforts should be made to increase our understanding of the effect of initiative policies and technological advancements on research quality.

Research Areas:
Tags:
Updated on
Backto the news archive

Related News

Spectrum Scholar Spotlight: Zhaneille Green

Thirteen iSchool master's students were named 2022-2023 Spectrum Scholars by the American Library Association (ALA) Office for Diversity, Literacy, and Outreach Services. This "Spectrum Scholar Spotlight" series highlights the School's scholars. Zhaneille Green holds a BA in geography and history from Clark University in Worcester, Massachusetts.

Zhaneille Green

Student award recipients announced

Each year, the School of Information Sciences recognizes a group of outstanding students for their achievement in academics as well as a number of attributes that contribute to professional success. Congratulations to this year's honorees!

Ly Dinh and Jessica Cheng

Ayorinde and Lopez receive CARLI Building Diversity Graduate Assistantships

The Consortium of Academic and Research Libraries in Illinois (CARLI) is pleased to announce that Elizabeth Ayorinde and Victor Lopez are the recipients of the Building Diversity Graduate Assistantships for the 2023-2024 academic year. The assistantships will provide experience, mentoring, and networking to participants with the goal of increasing the number of staff members from underrepresented groups at Illinois' two and four-year public and private college and university libraries.

Liu wins iSchool t-shirt design contest

Last month, Leep student Hui Liu was selected as the winner of the iSchool's t-shirt design contest. Her design was based on the prompt, "The Power of Information," which is also the School's tagline. Participants were asked to submit designs that illustrated the tagline from a student perspective.

Hui Liu wears a t-shirt with her winning design

Spectrum Scholar Spotlight: Daynali Flores-Rodriguez

Thirteen iSchool master's students were named 2022-2023 Spectrum Scholars by the American Library Association (ALA) Office for Diversity, Literacy, and Outreach Services. This "Spectrum Scholar Spotlight" series highlights the School's scholars. Daynali Flores-Rodriguez holds an MA and PhD in comparative and world literature from the University of Illinois Urbana-Champaign and BA in comparative literature from the University of Puerto Rico.

Daynali Flores-Rodriguez