Examining the Reproducibility of Meta-Analyses in Psychology

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: Meta-analyses are an important tool to evaluate the literature. Recent concerns about the reliability of single studies might also apply to meta-analyses, where the selection, inclusion, and coding of studies introduces substantial flexibility in the analysis and interpretation of results. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data comes in, or as novel statistical techniques (for example to correct for publication bias) are developed. Regrettably, research in medicine reveals meta-analyses often cannot be reproduced. We plan to quantify the reproducibility of meta-analyses in psychology by reproducing twenty published meta-analyses. We will examine how much variability in effect size estimates is observed in reproduced meta-analyses, both as a function of subjective choices, as when novel statistical techniques to correct for bias are applied to published meta-analyses. Difficulties in reproducing meta-analyses, or substantial variation between original and reproduced meta-analyses, would highlight the importance of increasing the transparency and reproducibility of meta-analyses. This project has the potential to initiate a discussion about the need to increase the transparency of published meta-analyses, and results may have important implications for journal policies.

Wiki

Meta-analysis coding instructions. 1) Only work on a meta-analysis you have been assigned to. For an overview of the meta-analyses you have been assigned to, see https://osf.io/suft6/ 2) The meta-analysis pdf files are available here: https://osf.io/q23ye/files/ in the folder “All Meta-Analyses”. There is also a link to the online PDF in the overview file here: https://osf.io/suft6/ 3) Most arti...

Files

Loading files...

Citation

Components

  • Study Selection


    Recent Activity

    Loading logs...

  • Examining the Reproducibility of Meta-Analyses in Psychology: A Preliminary Report

    Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to ev...

    Recent Activity

    Loading logs...

Recent Activity

Loading logs...

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message