The CREP is a crowdsourced replication project for undergraduate researchers. Six studies are available that are both highly cited and feasible for undergraduates to complete. Contributors who meet open science reporting guidelines receive small research awards (300-400 dollars) and when there are enough samples, contributors are encouraged to collaborate on a research paper. For the CREP completion steps, go to this subcomponent: https://osf.io/sskku/wiki/home/.
Purpose: Through student participation in large-scale replication efforts we aim to (1) facilitate student research training and (2) solidify research findings in psychological science.
Brief Summary: Replications are one key component of the scientific method (Asendorpf et al., 2013; IJzerman, Brandt, & Van Wolferen, 2013) and are an effective pedagogical tool (Frank & Saxe, 2012; Grahe et al., 2012). We aim to leverage these pedagogical benefits to promote replications of important findings. We have composed a list of current studies that we encourage students and instructors to replicate as part of research methods courses, independent studies, bachelor theses, and the like. Everyone who conducts a replication of a study on the list will be encouraged to submit their results to a centralized location where they can be further analyzed by interested researchers/experts. Students and instructors who conduct replications that become part of a published paper will be recognized as contributors on any publications that arise from the data. Student groups will share research credit rather than receive independent recognition. Any interested contributor is welcome to contribute to the writing process, and authorship order will be determined following APA guidelines with preference going to student and faculty contributors rather than project coordinators (i.e., Brandt, IJzerman, & Grahe). Project coordinators will ensure that data will not be lost, by monitoring progress of the projects.
From the perspective of an instructor, the CREP will provide instructors with a straightforward starting point for class projects and independent studies. It offers a very structured way to learn the mechanics of research, while still being challenged to think about the meaning of empirical studies. That is, in an attempt to stimulate creativity, we encourage replicators to think beyond the study, and come up with potential moderators or mediators. Importantly, replication studies lets students focus on learning the mechanics of research itself, not whether an outcome is significant. From the perspective of outside researchers, the CREP has the potential to provide a vast array of data for meta-analyses and other research syntheses. So beyond helping the students learn how to conduct research, it will help researchers understand theories in psychology.
Procedures: Research Methods and Capstone instructors are invited to contribute to this project by inviting their students to act as student researchers and acting as research sponsors. Contributors (students or other researchers) will conduct a replication of published psychological study (see list below). Contributors will obtain IRB approval from their home institution and collect their data when possible. Upon completion of the replication study, contributors will share statistics from the project necessary to complete meta-analyses (e.g., effect sizes). By sharing statistics, rather than data, many IRB problems regarding shared data are resolved. Ideally, replication projects will give students the opportunity to learn research methods by replicating important findings in psychology and help build a database of results surrounding some important psychological findings.
Any contributor can take the lead in completing an associated research report of the project. I.e., summarizing the total effect across replication attempts, searching for moderators of the effect size (e.g., is it stronger in Europe of the United States) etc. If multiple contributors are interested in taking the lead and the decision cannot be made amicably then Project Leaders will help adjudicate these disagreements. If no student contributor (or faculty mentor) wishes to write up the results for publication, the project leaders will do so.
The project center is on the Open Science Framework so that contributors can benefit from easy access to helpful resources such as IRB proposal guidance, materials, or power analyses from Project Leaders. We also recommend that when possible contributors follow the replication recipe for a convincing replication attempt, which is available here.
Sample Sizes: For each study (except for the Diener et al. study) we aim to collect 2.5 times the original samples size. In order to take part in the project, we expect replicators to at least collect the amount of participants in the original study. For the Diener et al. study, we expect replicators to at least contribute 100 participants in their own country sample.
Contributors: Research Methods and Capstone instructors are invited to use the studies list in their classes as an option for their students. Psi Chi members at any educational level are also invited to participate. Groups of students may wish to do so as a chapter activity or single students may wish to do so for a class research project. Students will need to find a faculty sponsor for their projects.
The List of Studies (see below for the complete list): Studies that were the top three cited empirical (non-meta-analysis) papers in each of the top journals for 9 sub-disciplines of psychology (according to the ISI, impact determined by eigenfactor.org) were the initial pool of studies for the list. These 27 studies were rated for feasibility by Dr. Mark Brandt and Dr. Hans IJzerman. The top 5 studies were chosen for the list. These studies represent well cited, but also recent empirical papers in psychological science. We encourage instructors to get in touch with us to suggest studies that they both consider classic in our field and feasible to conduct for students at the bachelor level.
Short Term Goals for the CREP: Spread the list of studies to be replicated, gather the relevant experimental materials needed to replicate the studies (when necessary), create an online hub for CREP using the OSF, and create a system for gathering information about the replication attempts as they are completed.
Long Term Goals for the CREP: Over the course of multiple years facilitate the research training of psychology students by encouraging replication projects and collecting data about the success (or failures) of the individual projects for use in meta-analyses and other research. We also feel that by creating this protocol, interaction about teaching methods of research is further encouraged. Finally, we hope replications to become a habit of psychology education and research.
Recommended Replication Studies.