Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: The unrealistic high rate of positive results within psychology increased the attention for replication research. Researchers who conduct a replication and want to statistically combine the results of their replication with a statistically significant original study encounter problems when using traditional meta-analysis techniques. The original study’s effect size is most probably overestimated because of it being statistically significant and this bias is not taken into consideration in traditional meta-analysis. We developed a hybrid method that does take statistical significance of the original study into account and enables (a) accurate effect size estimation, (b) estimation of a confidence interval, and (c) testing of the null hypothesis of no effect. We analytically approximate the performance of the hybrid method and describe its good statistical properties. Applying the hybrid method to the data of the Reproducibility Project Psychology (Open Science Collaboration, 2015) demonstrated that the conclusions based on the hybrid method are often in line with those of the replication, suggesting that many published psychological studies have smaller effect sizes than reported in the original study and that some effects may be even absent. We offer hands-on guidelines for how to statistically combine an original study and replication, and developed a web-based application (https://rvanaert.shinyapps.io/hybrid) for applying the hybrid method.

License: CC0 1.0 Universal

Files

Loading files...

Citation

Tags

Recent Activity

Loading logs...

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message