Main content

Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Project

Description: Null hypothesis significance testing (NSHT) is cited as a threat to validity and reproducibility. While many individuals suggest we focus on altering the p-value at which we deem an effect significant, we believe this suggestion is short-sighted. Alternative procedures (i.e., Bayesian analyses and Observation Oriented Modeling: OOM) can be more powerful and meaningful to our discipline. However, these methodologies are less frequently utilized and are rarely discussed in combination with NHST. Herein, we discuss three methodologies (NHST, Bayesian Model comparison, and OOM), then compare the possible interpretations of three analyses (ANOVA, Bayes Factor, and an Ordinal Pattern Analysis) in various data environments using a frequentist simulation study. We found that changing significance thresholds had little effect on conclusions. Further, we suggest that evaluating multiple estimates as evidence of an effect allows for more robust and nuanced interpretations of results and implies the need to redefine evidentiary value and reporting practices.

License: CC-By Attribution 4.0 International

Has supplemental materials for Beyond p-values: Utilizing Multiple Estimates to Evaluate Evidence on OSF Preprints


Add important information, links, or images here to describe your project.


Loading files...



Recent Activity

Loading logs...