As of March 2023, the School of Psychology at the University of Sussex is offering researchers the opportunity to have an independent statistician check the computational reproducibility of the results of papers before they are submitted for publication. Researchers are provided with a reproducibility report that they can reference in their manuscript. Each reproducibility report will be uploaded on this repository, and the DOI will be included in the report.
The reproducibility check is not meant as a substitute of peer review. The only aspect of this manuscript that has been examined is whether reanalysing the shared data using the same analysis methods as used by the authors yields the same results. In some cases, the reproducibility check may cover only some parts of the pre-processing steps and analyses outlined in the paper. The released statement outlines which steps of the analysis were submitted to a reproducibility check.
Readers and reviewers are still encouraged to fully evaluate all aspects of the reported study, such as the depth and scope of the literature review, the suitability of the experimental design to test the stated hypotheses, the suitability of the analysis method to analyse the data, the integrity of the data, and the conclusions reached by the authors. If any part of the analyses or experimental design of the study documented in the paper has been pre-registered, readers and reviewers should evaluate whether the pre-registration appropriately reduces researcher degrees of freedom, and whether the manuscript follows the methods and analyses steps outlined in the pre-registration.
This project is funded by the UKRI and it is entitled "Ensuring the computational reproducibility of to-be-submitted psychology papers".