Deciding what to replicate

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: Robust scientific knowledge is contingent upon replication of original findings. However, researchers who conduct replication studies face a difficult problem; there are many more studies in need of replication than there are funds available for replicating. To select studies for replication efficiently, we need to understand which studies are the most in need of replication. In other words, we need to understand which replication efforts have the highest expected utility. In this article we propose a general rule for study selection in replication research based on the replication value of the claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by replicating the claim, and is a function of (1) the value of being certain about the claim, and (2) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value.

License: CC-By Attribution 4.0 International

Wiki

Video talk on article https://www.youtube.com/watch?v=0j-nKDCPcRQ Related materials: What to Replicate? Justifications of study choice from 85 replication studies. (blog post): https://doi.org/10.5281/zenodo.1286715 Quantifying the corroboration of a finding (blog post): https://pedermisager.netlify.app/post/quantifying-the-corroboration-of-a-finding/

Files

Loading files...

Zotero

Loading citations...

Citation

Components

  • Unconf - What is worth replicating?

    Given that most research is original, and we have limited resources available for replication, we need guidelines for study selection in replication r...

    Recent Activity

    Loading logs...

Tags

Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.