Publication Bias Examined in Meta-Analyses from Psychology and Medicine: A Meta-Meta-Analysis

Contributors:

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias is widespread, how strongly it affects different scientific literatures is currently less well-known. We examined evidence of publication bias in a large-scale data set of primary studies that were included in of 83 meta-analyses published in Psychological Bulletin (representing meta-analyses from psychology) and 499 systematic reviews from the Cochrane Database of Systematic Reviews (representing meta-analyses from medicine). Publication bias was assessed on homogeneous subsets of theprimary studies included in meta-analyses, because publication bias methods do not have good statistical properties if the true effect size is heterogeneous. The rank-correlation test, Egger’s test, the test of excess significance, and p-uniform’s publication bias test yielded evidence for publication bias in approximately 10% of homogeneous subsets. Furthermore, we found hardly any evidence of overestimation of effect size because of publication bias, using the p-uniform method or when comparing the meta-analyses’ estimates with the estimates based on their largest studies. We therefore conclude that evidence for publication bias in the included meta-analyses is weak at best.

License: CC0 1.0 Universal

Files

Loading files...

Citation

Components

  • Pre-analysis plan


    Recent Activity

    Loading logs...

  • Monte-Carlo Simulation Study


    Recent Activity

    Loading logs...

Recent Activity

Loading logs...

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message