Date created: | Last Updated:

Category: Project

Description: Psychologists wrestle with how to best handle multiple comparisons, while maintaining a balance between false positives and false negatives. Undercorrection, such as ignoring the presence of multiple comparisons altogether, is known to yield an unacceptably high rate of false positives. Overcorrection, such as treating all tests as independent when they are not, results in overly conservative evaluations of statistical significance. This tutorial demonstrates $M_{eff}$ correction, a method for adjusting statistical significance thresholds for multiple comparisons, without the assumption of independence of tests. This method, in which the effective number of tests ($M_{eff}$) is estimated from the correlations among the variables being tested, was developed and validated in the field of genetics, but is based on statistical concepts (eigenvalues) that are very familiar to psychologists. $M_{eff}$ correction can be applied in psychological research to balance the necessity of correction for multiple comparisons with the concerns that arise from complex, correlated tests.