Statistical properties of methods based on the Q-statistic for estimating a confidence interval for the between-study variance in meta-analysis

Contributors:

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: The effect sizes of studies included in a meta-analysis do often not share a common true effect size due to differences in for instance the design of the studies. Estimates of this so-called between-study variance are usually imprecise. Hence, reporting a confidence interval together with a point estimate of the amount of between-study variance facilitates interpretation of the meta-analytic results. Two methods that are recommended to be used for creating such a confidence interval are the Q-profile and generalized Q-statistic method that both make use of the Q-statistic. These methods are exact if the assumptions underlying the random-effects model hold, but these assumptions are usually violated in practice such that confidence intervals of the methods are approximate rather than exact confidence intervals. We illustrate by means of two Monte-Carlo simulation studies with odds ratio as effect size measure that coverage probabilities of both methods can be substantially below the nominal coverage rate in situations that are representative for meta-analyses in practice. We also show that these too low coverage probabilities are caused by violations of the assumptions of the random-effects model (i.e., normal sampling distributions of the effect size measure and known sampling variances), and are especially prevalent if the sample sizes in the primary studies are small.

License: CC0 1.0 Universal

Files

Loading files...

Citation

Components

  • Monte-Carlo simulation study 1


    Recent Activity

    Loading logs...

  • Monte-Carlo simulation study 2


    Recent Activity

    Loading logs...

  • Supplemental materials heatmaps


    Recent Activity

    Loading logs...

  • Supplemental materials SMD


    Recent Activity

    Loading logs...

  • Supplemental materials RD


    Recent Activity

    Loading logs...

  • Example


    Recent Activity

    Loading logs...

Recent Activity

Loading logs...

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message