Main content

Date created: 2015-11-17 08:03 PM | Last Updated: 2017-07-09 05:50 PM

Category: Project

Description: Meta-analyses are an important tool to evaluate the literature. Recent concerns about the reliability of single studies might also apply to meta-analyses, where the selection, inclusion, and coding of studies introduces substantial flexibility in the analysis and interpretation of results. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data comes in, or as novel statistical techniques (for example to correct for publication bias) are developed. Regrettably, research in medicine reveals meta-analyses often cannot be reproduced. We plan to quantify the reproducibility of meta-analyses in psychology by reproducing twenty published meta-analyses. We will examine how much variability in effect size estimates is observed in reproduced meta-analyses, both as a function of subjective choices, as when novel statistical techniques to correct for bias are applied to published meta-analyses. Difficulties in reproducing meta-analyses, or substantial variation between original and reproduced meta-analyses, would highlight the importance of increasing the transparency and reproducibility of meta-analyses. This project has the potential to initiate a discussion about the need to increase the transparency of published meta-analyses, and results may have important implications for journal policies.

Wiki

Meta-analysis coding instructions.

  1. Only work on a meta-analysis you have been assigned to. For an overview of the meta-analyses you have been assigned to, see https://osf.io/suft6/

  2. The meta-analysis pdf files are available here: https://osf.io/q23ye/files/ in the folder “All Meta-Analyses”. There is also a link to the online PDF in the overview file here: https://osf.io/suft6/

  3. Most articles …

Files

Files can now be accessed and managed under the Files tab.

Citation

Components

Study Selection


Recent Activity

Loading logs...

Examining the Reproducibility of Meta-Analyses in Psychology: A Preliminary Report

Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to ev...

Recent Activity

Loading logs...

Recent Activity

Unable to retrieve logs at this time. Please refresh the page or contact support@osf.io if the problem persists.

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.