Date created: | Last Updated:

Category: Project

Description: Recent research indicates that Cohen’s d follows a non-central t-distribution, even as the central limit theorem approximates normal, contrary to the assumption that effects match the distribution of their test statistic (Cumming, 2012; Kelley, 2007; Smithson, 2003), and little research to date has focused on the variance overlap family of effects. Likewise, meta-analytic techniques also depend on accurate sampling variance estimations of effect sizes. To address this issue, we simulated 1,000 multivariate normal datasets for 1,152 combinations of varying sample sizes, standard deviations, levels, and level correlations. Eta, generalized eta (Olejnik & Algina 2003), omega, and all partial statistics were calculated for between and within subjects ANOVAs. Simulation results indicated an interactive pattern of mean effects and variances across conditions, with variance overlap measures reliably following a beta/gamma/normal distribution. We discuss the impact of design and data type on the measured effect and variance, as well as provide R scripts for simulations to adequately estimate sampling variance for meta-analyses.