Date created: | Last Updated:

Category: Analysis

Description: In September 2016, I received an automatic email from PubPeer (Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2015). The prevalence of statistical reporting errors in psychology (1985-2013). Behavior Research Methods. http://dx.doi.org/10.3758/s13428-015-0664-2) informing me that they were some statistical errors in the article. The putative wrong p values were related to a post-hoc analysis in the general discussion (meaning that there was no error in the main analyses of the two experiments). However, I decided to re-do all the analyses, to understand the errors. After a bit of archeological work, I found the initial DMDX files and re-ran all the analyses. The good news are that the inferential analyses led to the same conclusions as described in the paper (including the supposed wrong tests). Phew... The bad news are that the descriptive statistics are sometimes very different from what is described in the paper. I impute that to the use of different statistical software and to manual errors when preprocessing the data (I was used to process DMDX files manually with Excel) and analyzing the data (I was used to run them with STATISTICA). I'm therefore all the more happy to use R now for both pre-processing and statistical analyses (though it is still possible to make errors obviously). The script below presents the analyses I re-did, with highlights regarding conclusions.