Examining the Reproducibility of Meta-Analyses in Psychology

Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Project

Description: Meta-analyses are an important tool to evaluate the literature. Recent concerns about the reliability of single studies might also apply to meta-analyses, where the selection, inclusion, and coding of studies introduces substantial flexibility in the analysis and interpretation of results. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data comes in, or as novel statistical techniques (for example to correct for publication bias) are developed. Regrettably, research in medicine reveals meta-analyses often cannot be reproduced. We plan to quantify the reproducibility of meta-analyses in psychology by reproducing twenty published meta-analyses. We will examine how much variability in effect size estimates is observed in reproduced meta-analyses, both as a function of subjective choices, as when novel statistical techniques to correct for bias are applied to published meta-analyses. Difficulties in reproducing meta-analyses, or substantial variation between original and reproduced meta-analyses, would highlight the importance of increasing the transparency and reproducibility of meta-analyses. This project has the potential to initiate a discussion about the need to increase the transparency of published meta-analyses, and results may have important implications for journal policies.


Meta-analysis coding instructions. 1) Only work on a meta-analysis you have been assigned to. For an overview of the meta-analyses you have been assigned to, see https://osf.io/suft6/ 2) The meta-analysis pdf files are available here: https://osf.io/q23ye/files/ in the folder “All Meta-Analyses”. There is also a link to the online PDF in the overview file here: https://osf.io/suft6/ 3) Most arti...


Loading files...



  • Study Selection

    Recent Activity

    Loading logs...

  • Examining the Reproducibility of Meta-Analyses in Psychology: A Preliminary Report

    Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to ev...

    Recent Activity

    Loading logs...

Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.