Willis defends dissertation

Craig Willis
Craig Willis, Teaching Assistant Professor

Doctoral candidate Craig Willis successfully defended his dissertation, "Trust, But Verify: An Investigation of Methods of Verification and Dissemination of Computational Research Artifacts for Transparency and Reproducibility," on June 15.

His committee included Associate Professor Victoria Stodden (chair); Assistant Professor Peter Darch; Professor Bertram Ludäscher; and Michela Taufer, professor of electrical engineering and computer science at the University of Tennessee, Knoxville.

From the abstract: In this study, I investigate how research communities are addressing concerns about the quality and rigor of computational research. I focus specifically on initiatives to expand the peer review and publication process to include new requirements for the assessment and dissemination of computational research artifacts. I report the results of a multiple-case analysis of two primary (American Journal of Political Science, ACM/IEEE Supercomputing) and five supplemental cases in political science, computer science, economics, mathematics, and statistics. Cases were developed through qualitative analysis based on interviews with key stakeholders (n= 17) including editors, reviewers, and verifiers; a sample of (n= 27) verified artifacts; and documentary evidence including policies, guidelines, and workflows. Based on the cases, I identify key factors that influence the operationalization of policies and workflows; the elements that each community considers important to the assessment of computational transparency and reproducibility; as well as the tools and infrastructure that they leverage to aid in the creation, assessment and dissemination of reproducible research artifacts. I develop a framework to analyze the reproducibility initiatives and a conceptual model of reproducible research artifacts. I relate my findings to recommendations from the recent National Academies of Sciences, Engineering and Medicine (NASEM) report on Reproducibility and Replicability in Science and provide a set of normative guidelines for communities interested in pursuing similar initiatives with implications for journal and conference leadership; tool and infrastructure developers; and funding bodies. I conclude that, while promising, further efforts should be made to increase our understanding of the effect of initiative policies and technological advancements on research quality.

Research Areas:
Tags:
Updated on
Backto the news archive

Related News

iSchool undergraduates selected as 2024 Community Academic Scholars

The Interdisciplinary Health Sciences Institute (IHSI), in partnership with the Center for Social and Behavioral Science and with support from the Office of the Vice Chancellor for Diversity, Equity, & Inclusion, has selected BSIS students Sophie Chiewtrakoon and Madisen LeShoure as 2024 Community-Academic Scholars. The 17 scholars in this cohort represent seven colleges and schools and encompass a diverse array of fields of study, from community health to education to industrial design.  

Introductory course teaches information science concepts through game design

As part of Teaching Associate Professor Judith Pintar's Introduction to Information Sciences (IS 101) course, students are tasked with creating board games that teach various aspects of information science targeted to particular audiences. The students presented their creations on April 19 at the Game Studies and Design Spring 2024 Showcase. In addition to the game demonstrations, the event featured posters and presentations by students and faculty.

Spectrum Scholar Spotlight: Alyssa Brown

Seventeen iSchool master's students have been named 2023-2024 Spectrum Scholars by the American Library Association (ALA) Office for Diversity, Literacy, and Outreach Services. This "Spectrum Scholar Spotlight" series highlights the School's scholars. MSLIS student Alyssa Brown earned her BA in environmental studies from Middlebury College.

Alyssa Brown

iSchool researchers to present at ACM Web Conference

Members of Associate Professor Dong Wang's research group, the Social Sensing and Intelligence Lab, will present their research at the Web Conference 2024, which will be held from May 13-17 in Singapore. The Web Conference is the premier venue to present and discuss progress in research, development, standards, and applications of topics related to the Web.