[RP:P home][1] ## RP:P Process The Reproducibility Project: Psychology began in November 2011 as a community-based crowdsourcing project. The team maintained a substantial information commons and a defined protocol for replication teams to follow. These are the main resources for reviewing the project history and documentation. ### Group Communication - **OSF Discussion Group**: The public Open Science Framework discussion group served as the origin of the Reproducibility Project: Psychology. There, the team developed and discussed and implemented the project design. The history of those discussions is available [here][2] from November 2011 through January 2015. In late 2014, project discussions moved to a RP:P team list. ### Project Logistics - **Claiming Articles for Replication**: Scientists interested in joining the project could review information about each of the articles in the available pool to help them select a study that best matched their interests, resources, and expertise. Some RP:P team members were recruited directly by project coordinators. [This document][3] represents the available study pool at the conclusion of data collection for all 100 experiments included in the *Science* publication. - **Researcher Guide**: The main project protocol is available [here][4]. This guide covers obtaining materials, planning analyses, writing and reviewing a pre-data collection report, reporting findings, and a final report review. This guide was a living document, first instantiated as a Google Doc and then transferred to the OSF as a series of wiki pages. The guide was revised to include additional details and links to resources as the volunteer group grew. Requirements for participation (e.g. minimum power requirements and pre-data collection reports) did not change. - **Data Reporting**: Researchers were responsible for reporting their progress and findings. Until June 2014, researchers recorded their data into a spreadsheet, following the instructions provided in the researcher guide. After June 2014, researchers reported their findings by responding to a survey. Project Coordinators logged responses into a master data file. Additional surveys were distributed to measure characteristics of the replications and teams. These surveys can be found in the [OSF Storage of the Analysis component][5] of this project. - **Project Review Process**: Following the completion of a replication, the project coordinators and other volunteers confirmed that the replication had all required elements (e.g. public data and materials, a replication report, complete contributor list). Documentation of this review is available [here][6]. - **Analysis Audit**: The key effect from each replication was audited with a re-analysis by other members of the project team. These team members wrote their analyses in R, following a template to improve reproducibility. (Replication teams could use their preferred data analysis package.) Documentation of the template and instructions for the analysis audit is available [here][7]. ### Reporting the Reproducibility Project: Psychology - **Initial Press Report**: In March 2012, Science published the first media story about the Reproducibility Project: Psychology. Many others have been written, but that introduction was quite effective in describing the project goals, design, and motivation at its outset. This article is available [here][8]. - **Project Introduction Article**: In November 2012, an introduction to the Reproducibility Project: Psychology was published in *Perspectives on Psychological Science*. This article summarized the project motivation and what we expected to learn from the effort. A copy is available [here][9]. - **Project Design Chapter**: In 2014, the team published a chapter in *Implementing Reproducible Research* (Stodden, Leisch, Peng, 2014) about the project design and mechanisms for quality control. A copy of the chapter is available [here][10]. - **Published Findings**: In August of 2015, "Estimating the Reproducibility of Psychological Science" was published in *Science*. A copy of the manuscript can be found [here][11]. Guides to the analyses are available [here][12]. [1]: https://osf.io/ezcuj/wiki/home/ [2]: https://groups.google.com/forum/#!forum/openscienceframework [3]: https://osf.io/tagj9/ [4]: https://osf.io/ru689/ [5]: https://osf.io/ytpuq/files/ [6]: https://osf.io/xyjvc/ [7]: https://osf.io/nefzv/ [8]: http://www.sciencemag.org/content/335/6076/1558 [9]: https://osf.io/sejcv/ [10]: https://osf.io/9h47z/ [11]: https://osf.io/phtye/ [12]: https://osf.io/ytpuq/wiki/home/