Home

Menu

Loading wiki pages...

View
Wiki Version:
<p><a href="https://osf.io/ezcuj/wiki/home/" rel="nofollow">RP:P home</a></p> <h2>RP:P Process</h2> <p>The Reproducibility Project: Psychology began in November 2011 as a community-based crowdsourcing project. The team maintained a substantial information commons and a defined protocol for replication teams to follow. These are the main resources for reviewing the project history and documentation.</p> <h3>Group Communication</h3> <ul> <li><strong>OSF Discussion Group</strong>: The public Open Science Framework discussion group served as the origin of the Reproducibility Project: Psychology. There, the team developed and discussed and implemented the project design. The history of those discussions is available <a href="https://groups.google.com/forum/#!forum/openscienceframework" rel="nofollow">here</a> from November 2011 through January 2015. In late 2014, project discussions moved to a RP:P team list. </li> </ul> <h3>Project Logistics</h3> <ul> <li><strong>Claiming Articles for Replication</strong>: Scientists interested in joining the project could review information about each of the articles in the available pool to help them select a study that best matched their interests, resources, and expertise. Some RP:P team members were recruited directly by project coordinators. <a href="https://osf.io/tagj9/" rel="nofollow">This document</a> represents the available study pool at the conclusion of data collection for all 100 experiments included in the <em>Science</em> publication.</li> <li><strong>Researcher Guide</strong>: The main project protocol is available <a href="https://osf.io/ru689/" rel="nofollow">here</a>. This guide covers obtaining materials, planning analyses, writing and reviewing a pre-data collection report, reporting findings, and a final report review. This guide was a living document, first instantiated as a Google Doc and then transferred to the OSF as a series of wiki pages. The guide was revised to include additional details and links to resources as the volunteer group grew. Requirements for participation (e.g. minimum power requirements and pre-data collection reports) did not change. </li> <li><strong>Data Reporting</strong>: Researchers were responsible for reporting their progress and findings. Until June 2014, researchers recorded their data into a spreadsheet, following the instructions provided in the researcher guide. After June 2014, researchers reported their findings by responding to a survey. Project Coordinators logged responses into a master data file. Additional surveys were distributed to measure characteristics of the replications and teams. These surveys can be found in the <a href="https://osf.io/ytpuq/files/" rel="nofollow">OSF Storage of the Analysis component</a> of this project.</li> <li><strong>Project Review Process</strong>: Following the completion of a replication, the project coordinators and other volunteers confirmed that the replication had all required elements (e.g. public data and materials, a replication report, complete contributor list). Documentation of this review is available <a href="https://osf.io/xyjvc/" rel="nofollow">here</a>.</li> <li><strong>Analysis Audit</strong>: The key effect from each replication was audited with a re-analysis by other members of the project team. These team members wrote their analyses in R, following a template to improve reproducibility. (Replication teams could use their preferred data analysis package.) Documentation of the template and instructions for the analysis audit is available <a href="https://osf.io/nefzv/" rel="nofollow">here</a>.</li> </ul> <h3>Reporting the Reproducibility Project: Psychology</h3> <ul> <li><strong>Initial Press Report</strong>: In March 2012, Science published the first media story about the Reproducibility Project: Psychology. Many others have been written, but that introduction was quite effective in describing the project goals, design, and motivation at its outset. This article is available <a href="http://www.sciencemag.org/content/335/6076/1558" rel="nofollow">here</a>.</li> <li><strong>Project Introduction Article</strong>: In November 2012, an introduction to the Reproducibility Project: Psychology was published in <em>Perspectives on Psychological Science</em>. This article summarized the project motivation and what we expected to learn from the effort. A copy is available <a href="https://osf.io/sejcv/" rel="nofollow">here</a>.</li> <li><strong>Project Design Chapter</strong>: In 2014, the team published a chapter in <em>Implementing Reproducible Research</em> (Stodden, Leisch, Peng, 2014) about the project design and mechanisms for quality control. A copy of the chapter is available <a href="https://osf.io/9h47z/" rel="nofollow">here</a>.</li> <li><strong>Published Findings</strong>: In August of 2015, "Estimating the Reproducibility of Psychological Science" was published in <em>Science</em>. A copy of the manuscript can be found <a href="https://osf.io/phtye/" rel="nofollow">here</a>. Guides to the analyses are available <a href="https://osf.io/ytpuq/wiki/home/" rel="nofollow">here</a>.</li> </ul>
OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
Accept
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.