Main content



Loading wiki pages...

Wiki Version:
[RP:P home][1] ## RP:P Process The Reproducibility Project: Psychology began in November 2011 as a community-based crowdsourcing project. The team maintained a substantial information commons and a defined protocol for replication teams to follow. These are the main resources for reviewing the project history and documentation. ### Group Communication - **OSF Discussion Group**: The public Open Science Framework discussion group served as the origin of the Reproducibility Project: Psychology. There, the team developed and discussed and implemented the project design. The history of those discussions is available [here][2] from November 2011 through January 2015. In late 2014, project discussions moved to a RP:P team list. ### Project Logistics - **Claiming Articles for Replication**: Scientists interested in joining the project could review information about each of the articles in the available pool to help them select a study that best matched their interests, resources, and expertise. Some RP:P team members were recruited directly by project coordinators. [This document][3] represents the available study pool at the conclusion of data collection for all 100 experiments included in the *Science* publication. - **Researcher Guide**: The main project protocol is available [here][4]. This guide covers obtaining materials, planning analyses, writing and reviewing a pre-data collection report, reporting findings, and a final report review. This guide was a living document, first instantiated as a Google Doc and then transferred to the OSF as a series of wiki pages. The guide was revised to include additional details and links to resources as the volunteer group grew. Requirements for participation (e.g. minimum power requirements and pre-data collection reports) did not change. - **Data Reporting**: Researchers were responsible for reporting their progress and findings. Until June 2014, researchers recorded their data into a spreadsheet, following the instructions provided in the researcher guide. After June 2014, researchers reported their findings by responding to a survey. Project Coordinators logged responses into a master data file. Additional surveys were distributed to measure characteristics of the replications and teams. These surveys can be found in the [OSF Storage of the Analysis component][5] of this project. - **Project Review Process**: Following the completion of a replication, the project coordinators and other volunteers confirmed that the replication had all required elements (e.g. public data and materials, a replication report, complete contributor list). Documentation of this review is available [here][6]. - **Analysis Audit**: The key effect from each replication was audited with a re-analysis by other members of the project team. These team members wrote their analyses in R, following a template to improve reproducibility. (Replication teams could use their preferred data analysis package.) Documentation of the template and instructions for the analysis audit is available [here][7]. ### Reporting the Reproducibility Project: Psychology - **Initial Press Report**: In March 2012, Science published the first media story about the Reproducibility Project: Psychology. Many others have been written, but that introduction was quite effective in describing the project goals, design, and motivation at its outset. This article is available [here][8]. - **Project Introduction Article**: In November 2012, an introduction to the Reproducibility Project: Psychology was published in *Perspectives on Psychological Science*. This article summarized the project motivation and what we expected to learn from the effort. A copy is available [here][9]. - **Project Design Chapter**: In 2014, the team published a chapter in *Implementing Reproducible Research* (Stodden, Leisch, Peng, 2014) about the project design and mechanisms for quality control. A copy of the chapter is available [here][10]. - **Published Findings**: In August of 2015, "Estimating the Reproducibility of Psychological Science" was published in *Science*. A copy of the manuscript can be found [here][11]. Guides to the analyses are available [here][12]. [1]: [2]:!forum/openscienceframework [3]: [4]: [5]: [6]: [7]: [8]: [9]: [10]: [11]: [12]:
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.