Beyond p-values: Utilizing Multiple Estimates to Evaluate Evidence

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: Null hypothesis significance testing (NSHT) is cited as a threat to validity and reproducibility. While many individuals suggest we focus on altering the p-value at which we deem an effect significant, we believe this suggestion is short-sighted. Alternative procedures (i.e., Bayesian analyses and Observation Oriented Modeling: OOM) can be more powerful and meaningful to our discipline. However, these methodologies are less frequently utilized and are rarely discussed in combination with NHST. Herein, we discuss three methodologies (NHST, Bayesian Model comparison, and OOM), then compare the possible interpretations of three analyses (ANOVA, Bayes Factor, and an Ordinal Pattern Analysis) in various data environments using a frequentist simulation study. We found that changing significance thresholds had little effect on conclusions. Further, we suggest that evaluating multiple estimates as evidence of an effect allows for more robust and nuanced interpretations of results and implies the need to redefine evidentiary value and reporting practices.

License: CC-By Attribution 4.0 International

Has supplemental materials for Beyond p-values: Utilizing Multiple Estimates to Evaluate Evidence on OSF Preprints

Wiki

View the Shiny Application: go here. Notes on how to recreate our work: - To recreate the manuscript, you will need to download the manuscript files folder and run the Rmarkdown within that folder (you can open this file with RStudio). The markdown includes the simulation code that created the files in the output folder. - We suggest starting by searching for "library" within our document to inst...

Files

Loading files...

Citation

Tags

Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.