Social Sciences Replication Project =================================== --- This OSF repository accompanies the publication Camerer, C., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B., Pfeiffer, T., Altmejd, A., Buttrick, N., Chan, T., Chen, Y., Forsell, E., Gampa, A., Heikenstein, E., Hummer, L., Imai, T., Isaksson, S., Manfredi, D., Rose, J., Wagenmakers, E., and Wu, H. (2018). *Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015*, **Nature Human Behavior**, in press. and contains all data and materials reported in the article and supplementary information. In particular, the repository consists of 22 independent sub-repositories; one for each of the 21 studies replicated and one for the aggregate results reported in the manuscript and the supplementary information (see below). --- ### Sub-repositories for each replication study. Each of these repositories is subdivided in three components: (i) Replication Reports, (ii) Study Materials, and (iii) Data & Analysis. * The **“Replication Reports”** component contains two versions of the replication and analysis protocol: one version posted and pre-registered before any data has been collected, and a second version including the analyses and results of the replication data. * The **“Study Materials”** component contains all materials, protocols, procedures, experimental software, etc. (as applicable) that were used to conduct the replication experiments. * The **“Data & Analysis”** component contains any data obtained from the replication experiments and the script files used to analyze the data. --- ### Sub-repository for aggregate data and analysis. The sub-repository **“[Materials][1]”** contains all data and analysis scripts that are required to reproduce the aggregate results reported in the manuscript and the supplementary information. In particular, the repository is divided into the following components: * The **“[Replication Reports][2]”** component contains all 21 final replication reports of the individual studies replicated including the results. The reports posted in this component are identical to the replication reports (post-data collection) available in the “Replication Reports” component in each of the individual study repositories. * The **“[Data][3]”** component consists of two sub-folders, “Data Raw” and “Data Processed”. *“Data Raw”* contains the raw data of prediction markets, surveys, comparative results from the “Experimental Economics Replication Project” (EERP; Camerer et al. 2016) and the “Reproducibility Project: Psychology” (RPP; Open Science Collaboration, 2015), as well as data obtained from the Bayesian Analysis. The *“Data Processed”* component contains all data files created using the scripts in the folder *“01 – DataProcessing”* of the “[Scripts][4]” component. The number labeling is identical to the labeling of the corresponding scripts creating the particular data file. * The **“[Scripts][5]”** component contains all Stata \*.do-files used for the aggregate data analysis and the creation of figures and tables. The file `01 – Programs.do` in the folder *“00 – Programs”* consists of all Stata “programs” (i.e. user-written functions) to determine standardized effect sizes, confidence intervals, p-values, prediction intervals, small effects (Small Telescope Apporach), fixed-effect weighted meta-effects, replication sample sizes, and statistical power of the replication experiments. These functions are called from the particular scripts in the *“01 – Data Processing”* folder to generate the data files in the “[Data][6]” component. The folders *“02 – Analysis”* contains two scripts used for the analyses reported in the manuscript and the supplementary materials, respectively. The output of the scripts is stored as a log-file to the “[Results][7]” component. All results are logged in (approximately) the same order as reported in the manuscript and the supplementary materials, respectively. The folders *“03 – Figures”* and *“04 – Tables”* contain the scripts used to generate the figures and tables stored in the “[Figures][8]” and “[Tables][9]” component, respectively. The tables are exported as \*.pdf-files by calling LaTeX via the shell-extension of Stata. To reproduce the tables, thus, an installation of an arbitrary LaTeX compiler (add to the system’s `PATH` environmental variable) is required. * The **“[Bayesian Analysis][10]”** component contains three \*.pdf-files outlining the procedures and methods used to determine the Bayesian default and replication factors and details on the implementation of the Bayesian mixture model. The analysis scripts for the individual studies are contained in the folder *“Analysis: Individual Studies”*; aggregate analysis scripts can be found in the *“Analysis”* folder. * The **“[Results][11]”** component contains the results of the aggregate data analysis for the main text and the supplementary materials as \*.log- and \*.smcl-files (the latter can be opened with Stata to reproduce the Stata output format). * The **“[Figures][12]”** component contains the four figures included in the manuscript and the nine figures included in the supplementary materials in \*.png-format (any other data format can be obtained by running the corresponding script in the folder *“03 – Figures”* in the “[Scripts][13]” component. * The **“[Tables][14]”** component contains the seven tables reported in the supplementary materials in \*.pdf-format as generated by the corresponding scripts in the folder *“04 – Tables”* in the “[Scripts][15]” component. * The **“[Complete Repository (\*.zip)][16]”** component consists of a single zipped file (`Analysis.zip`), containing all the data and materials described above (except for the “[Bayesian Analysis][10]” component). As all data imports and exports in the scripts are based on relative paths, any of the scripts is executable without adapting folder structures, etc. The only requirement to reproduce any result, analysis, figure, or table is a working installation of Stata and LaTeX (for reproduction of the Tables). Some of the scripts make use of user-written commands and programs that cannot be installed on the fly. In case, one of the scripts does not execute properly, please install the missing command using `findit [command name]` in Stata’s command line. --- [1]: https://osf.io/z5unf/ [2]: https://osf.io/sbru6/ [3]: https://osf.io/x4ca2/ [4]: https://osf.io/vztcn/ [5]: https://osf.io/vztcn/ [6]: https://osf.io/x4ca2/ [7]: https://osf.io/3pc4h/ [8]: https://osf.io/8hzms/ [9]: https://osf.io/6erp3/ [10]: https://osf.io/956xg/ [11]: https://osf.io/3pc4h/ [12]: https://osf.io/8hzms/ [13]: https://osf.io/vztcn/ [14]: https://osf.io/6erp3/ [15]: https://osf.io/vztcn/ [16]: https://osf.io/y4u3h/