Publication Bias Examined in Meta-Analyses from Psychology and Medicine: A Meta-Meta-Analysis


Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Project

Description: Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias is widespread, how strongly it affects different scientific literatures is currently less well-known. We examined evidence of publication bias in a large-scale data set of primary studies that were included in of 83 meta-analyses published in Psychological Bulletin (representing meta-analyses from psychology) and 499 systematic reviews from the Cochrane Database of Systematic Reviews (representing meta-analyses from medicine). Publication bias was assessed on homogeneous subsets of theprimary studies included in meta-analyses, because publication bias methods do not have good statistical properties if the true effect size is heterogeneous. The rank-correlation test, Egger’s test, the test of excess significance, and p-uniform’s publication bias test yielded evidence for publication bias in approximately 10% of homogeneous subsets. Furthermore, we found hardly any evidence of overestimation of effect size because of publication bias, using the p-uniform method or when comparing the meta-analyses’ estimates with the estimates based on their largest studies. We therefore conclude that evidence for publication bias in the included meta-analyses is weak at best.

License: CC0 1.0 Universal


Loading files...



  • Pre-analysis plan

    Recent Activity

    Loading logs...

  • Monte-Carlo Simulation Study

    Recent Activity

    Loading logs...

Recent Activity

Loading logs...

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message