CFG Seminar by Yuanxi Fu: Assessing trust and stability in evidence synthesis: a network analysis approach

Abstract: Trust and stability are two major concerns facing evidence synthesis. Selection bias, sponsor bias, and author/affiliation-related bias impair trust in evidence production and synthesis. Unstable evidence synthesis means different reviews published around the same time come to different conclusions, which risks immature decision-making. In this talk, I will present our research that applies network analysis techniques to assess trust and stability in evidence synthesis. First, I will discuss the insights generated from our study of the salt controversy (Hsiao et al., 2020), an example of unstable evidence synthesis. Inspired by this study and Prof. Jodi Schneider’s prior work (Jackson & Schneider, 2018), I will discuss the potential of situating trust and stability in the Toulmin model (Toulmin, 2003). Second, I will discuss our work on the claim-specific network. Claim-specific network is a network construct that comprises all publications addressing a research question and their citation relationships. We proposed a new network metric, the ratio between the real and expected citation count, to select “marginalized papers” (Fu et al., 2021; Wan et al., 2021). By examining how well an article cites marginalized papers, we may detect potential selection bias. I will present a case study to demonstrate this method and point out challenges in applying it in reality.

The Conceptual Foundations Group (CFG) is an interest-based research group, centered around clarifying foundational concepts relating to information organization, data curation, and semantic technologies.

Questions? Contact Lan Li