[RP:P home][1]
## RP:P Process
The Reproducibility Project: Psychology began in November 2011 as a community-based crowdsourcing project. The team maintained a substantial information commons and a defined protocol for replication teams to follow. These are the main resources for reviewing the project history and documentation.
### Group Communication
- **OSF Discussion Group**: The public Open Science Framework discussion group served as the origin of the Reproducibility Project: Psychology. There, the team developed and discussed and implemented the project design. The history of those discussions is available [here][2] from November 2011 through January 2015. In late 2014, project discussions moved to a RP:P team list.
### Project Logistics
- **Claiming Articles for Replication**: Scientists interested in joining the project could review information about each of the articles in the available pool to help them select a study that best matched their interests, resources, and expertise. Some RP:P team members were recruited directly by project coordinators. [This document][3] represents the available study pool at the conclusion of data collection for all 100 experiments included in the *Science* publication.
- **Researcher Guide**: The main project protocol is available [here][4]. This guide covers obtaining materials, planning analyses, writing and reviewing a pre-data collection report, reporting findings, and a final report review. This guide was a living document, first instantiated as a Google Doc and then transferred to the OSF as a series of wiki pages. The guide was revised to include additional details and links to resources as the volunteer group grew. Requirements for participation (e.g. minimum power requirements and pre-data collection reports) did not change.
- **Data Reporting**: Researchers were responsible for reporting their progress and findings. Until June 2014, researchers recorded their data into a spreadsheet, following the instructions provided in the researcher guide. After June 2014, researchers reported their findings by responding to a survey. Project Coordinators logged responses into a master data file. Additional surveys were distributed to measure characteristics of the replications and teams. These surveys can be found in the [OSF Storage of the Analysis component][5] of this project.
- **Project Review Process**: Following the completion of a replication, the project coordinators and other volunteers confirmed that the replication had all required elements (e.g. public data and materials, a replication report, complete contributor list). Documentation of this review is available [here][6].
- **Analysis Audit**: The key effect from each replication was audited with a re-analysis by other members of the project team. These team members wrote their analyses in R, following a template to improve reproducibility. (Replication teams could use their preferred data analysis package.) Documentation of the template and instructions for the analysis audit is available [here][7].
### Reporting the Reproducibility Project: Psychology
- **Initial Press Report**: In March 2012, Science published the first media story about the Reproducibility Project: Psychology. Many others have been written, but that introduction was quite effective in describing the project goals, design, and motivation at its outset. This article is available [here][8].
- **Project Introduction Article**: In November 2012, an introduction to the Reproducibility Project: Psychology was published in *Perspectives on Psychological Science*. This article summarized the project motivation and what we expected to learn from the effort. A copy is available [here][9].
- **Project Design Chapter**: In 2014, the team published a chapter in *Implementing Reproducible Research* (Stodden, Leisch, Peng, 2014) about the project design and mechanisms for quality control. A copy of the chapter is available [here][10].
- **Published Findings**: In August of 2015, "Estimating the Reproducibility of Psychological Science" was published in *Science*. A copy of the manuscript can be found [here][11]. Guides to the analyses are available [here][12].
[1]: https://osf.io/ezcuj/wiki/home/
[2]: https://groups.google.com/forum/#!forum/openscienceframework
[3]: https://osf.io/tagj9/
[4]: https://osf.io/ru689/
[5]: https://osf.io/ytpuq/files/
[6]: https://osf.io/xyjvc/
[7]: https://osf.io/nefzv/
[8]: http://www.sciencemag.org/content/335/6076/1558
[9]: https://osf.io/sejcv/
[10]: https://osf.io/9h47z/
[11]: https://osf.io/phtye/
[12]: https://osf.io/ytpuq/wiki/home/