Willis defends dissertation

Craig Willis
Craig Willis, Teaching Assistant Professor

Doctoral candidate Craig Willis successfully defended his dissertation, "Trust, But Verify: An Investigation of Methods of Verification and Dissemination of Computational Research Artifacts for Transparency and Reproducibility," on June 15.

His committee included Associate Professor Victoria Stodden (chair); Assistant Professor Peter Darch; Professor Bertram Ludäscher; and Michela Taufer, professor of electrical engineering and computer science at the University of Tennessee, Knoxville.

From the abstract: In this study, I investigate how research communities are addressing concerns about the quality and rigor of computational research. I focus specifically on initiatives to expand the peer review and publication process to include new requirements for the assessment and dissemination of computational research artifacts. I report the results of a multiple-case analysis of two primary (American Journal of Political Science, ACM/IEEE Supercomputing) and five supplemental cases in political science, computer science, economics, mathematics, and statistics. Cases were developed through qualitative analysis based on interviews with key stakeholders (n= 17) including editors, reviewers, and verifiers; a sample of (n= 27) verified artifacts; and documentary evidence including policies, guidelines, and workflows. Based on the cases, I identify key factors that influence the operationalization of policies and workflows; the elements that each community considers important to the assessment of computational transparency and reproducibility; as well as the tools and infrastructure that they leverage to aid in the creation, assessment and dissemination of reproducible research artifacts. I develop a framework to analyze the reproducibility initiatives and a conceptual model of reproducible research artifacts. I relate my findings to recommendations from the recent National Academies of Sciences, Engineering and Medicine (NASEM) report on Reproducibility and Replicability in Science and provide a set of normative guidelines for communities interested in pursuing similar initiatives with implications for journal and conference leadership; tool and infrastructure developers; and funding bodies. I conclude that, while promising, further efforts should be made to increase our understanding of the effect of initiative policies and technological advancements on research quality.

Research Areas:
Tags:
Updated on
Backto the news archive

Related News

Wang group to present at BigData 2024

Members of Associate Professor Dong Wang's research group, the Social Sensing and Intelligence Lab, will present their research at the 2024 IEEE International Conference on Big Data (BigData 2024), which will be held from December 15-18 in Washington, D.C. BigData 2024 is the premier venue to present and discuss progress in research, development, standards, and applications of topics in artificial intelligence, machine learning and big data analytics.

Dong Wang

Walters learns history of ATO through archives assistantship

When MSLIS student Deborah Walters was offered a graduate assistantship to work in the Alpha Tau Omega Archives, she viewed it as a "unique opportunity to have a hands-on independent experience in archives" that she couldn't pass up. Alpha Tau Omega (ATO) is a social fraternity that was founded at the Virginia Military Institute in 1865. Its archives are among the national fraternity collections housed at the Student Life and Culture Archives at the University of Illinois.

Deborah Walters

Antwi grateful for Balz Scholarship

MSLIS student Victora Antwi is grateful for the financial support that she has received through the Balz Endowment Fund. An international student from the Mampong-Nsuta in the Ashanti Region, Ghana, Antwi earned her bachelor’s degree in information studies in 2020 from the University of Ghana. 

Victoria Antwi

Illinois researchers examine teens’ use of generative AI, safety concerns

Teenagers use generative artificial intelligence for many purposes, including emotional support and social interactions. A study by University of Illinois Urbana-Champaign researchers found that parents have little understanding of GAI, how their children use it and its potential risks, and that GAI platforms offer insufficient protection to ensure children’s safety.

Yang Wang

Bell receives Fulbright-Hays Fellowship for dissertation fieldwork in Brazil

Little did doctoral candidate Kainen Bell know in 2013 when he was an undergraduate studying abroad in Brazil that the country would play a major role in his future dissertation research. Since his first trip, he has returned to Brazil multiple times, even completing a Fulbright study and working for a community-based organization in the country. Now, Bell is preparing to return again, this time to spend ten months conducting research as a recipient of the prestigious Fulbright-Hays Doctoral Dissertation Research Abroad (DDRA) Fellowship.

Kainen Bell