Date created: | Last Updated:

Category: Uncategorized

Description: In meta-analyses, selective reporting arises when some results within a study or the entire study are more likely to be reported due to certain properties such as statistical significance or effect size magnitude. Selective reporting can result in over-estimated average effect size, inflated Type-I error rates, and inappropriate inferences about the intervention effects. In practice, meta-analysts seek to both identify the presence of selective reporting and adjust for the distortions it creates. Various statistical methods have been developed for detecting and correcting selection bias. However, these methods are usually based on the assumption that effect sizes are independent, which is violated in meta-analysis with multiple, dependent effect sizes. In this paper, we evaluate how different selective reporting adjustment methods work in the presence of effect size dependencies. Moreover, we propose novel adaptations of several adjustment methods based on a multivariate working model and weighting scheme that correct for selective reporting bias while handling dependencies among effect sizes. We investigate the performance of existing adjustment methods and our novel adaptations via a comprehensive simulation study of dependent effect size estimates. We focus on the estimation of overall average effects and examine performance criteria including bias, RMSE, and confidence interval coverage rate and width. Based on the simulation findings, we provide suggestions on how to correct selective reporting bias while accounting for effect size dependencies.