Willis defends dissertation

Craig Willis
Craig Willis, Teaching Assistant Professor

Doctoral candidate Craig Willis successfully defended his dissertation, "Trust, But Verify: An Investigation of Methods of Verification and Dissemination of Computational Research Artifacts for Transparency and Reproducibility," on June 15.

His committee included Associate Professor Victoria Stodden (chair); Assistant Professor Peter Darch; Professor Bertram Ludäscher; and Michela Taufer, professor of electrical engineering and computer science at the University of Tennessee, Knoxville.

From the abstract: In this study, I investigate how research communities are addressing concerns about the quality and rigor of computational research. I focus specifically on initiatives to expand the peer review and publication process to include new requirements for the assessment and dissemination of computational research artifacts. I report the results of a multiple-case analysis of two primary (American Journal of Political Science, ACM/IEEE Supercomputing) and five supplemental cases in political science, computer science, economics, mathematics, and statistics. Cases were developed through qualitative analysis based on interviews with key stakeholders (n= 17) including editors, reviewers, and verifiers; a sample of (n= 27) verified artifacts; and documentary evidence including policies, guidelines, and workflows. Based on the cases, I identify key factors that influence the operationalization of policies and workflows; the elements that each community considers important to the assessment of computational transparency and reproducibility; as well as the tools and infrastructure that they leverage to aid in the creation, assessment and dissemination of reproducible research artifacts. I develop a framework to analyze the reproducibility initiatives and a conceptual model of reproducible research artifacts. I relate my findings to recommendations from the recent National Academies of Sciences, Engineering and Medicine (NASEM) report on Reproducibility and Replicability in Science and provide a set of normative guidelines for communities interested in pursuing similar initiatives with implications for journal and conference leadership; tool and infrastructure developers; and funding bodies. I conclude that, while promising, further efforts should be made to increase our understanding of the effect of initiative policies and technological advancements on research quality.

Research Areas:
Tags:
Updated on
Backto the news archive

Related News

iSchool researchers to present at CHI 2025

iSchool faculty and students will present their research at the ACM Conference on Human Factors in Computing Systems (CHI 2025), which will be held from April 26 to May 1 in Yokohama, Japan. 

Kemboi receives the Research and Advocacy Social Justice Award

PhD student Gladys Kemboi has received the 2025 Research and Advocacy Social Justice Award from the Office of Diversity & Social Justice Education in the Office of Student Affairs. She was presented with the award at the Social Justice Awards Ceremony, which was held on April 8 in the Illini Union. The annual event honors and celebrates the work and dedication of University of Illinois community members seeking to create a more inclusive and equitable campus.

Gladys Kemboi

Undergraduate Research Symposium features iSchool students and mentors

Several iSchool undergraduate students will participate in the 18th annual Undergraduate Research Symposium. During the event, visitors will learn about undergraduate research projects through oral and poster presentations, creative performances, and art exhibits. All are welcome to attend the symposium, which will be held on April 24 from 9:00 a.m.-5:00 p.m. in the Illini Rooms and South Lounge of the Illini Union. Oral presentations will be held on the second floor of the Illini Union.

Wang wins grand prize at Research Live!

Informatics PhD student Olivia Wang won the Grand Prize at the 2025 Research Live! competition, which was held on April 8 in the Campus Instructional Facility Atrium. At the event, which is hosted by the Graduate College, thirteen finalists presented their graduate research in three minutes or less to a general audience. Wang received $500 as the Grand Prize winner.

Olivia Wang