Documentation ============= This project site contains data and materials for [a research study published in Behavior Research Methods](https://doi.org/10.3758/s13428-023-02164-8). Here we provide documentation and explanation of the contents of this project site. If you are viewing this from the main page of the project site, click [here](https://osf.io/yrbhe/wiki), or click "Read More" below, to view full documentation. @[toc] # Replication # The current study is a preregistered replication of [McDaniel, Wildman, & Anderson (2012)](https://doi.org/10.1016/j.jarmac.2011.10.001), full citation below: > McDaniel, M. A., Wildman, K. M., & Anderson, J. L. (2012). Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study. *Journal of Applied Research in Memory and Cognition, 1*(1), 18–26. https://doi.org/10.1016/j.jarmac.2011.10.001 # Terracotta # The current research study is presented as a demonstration of [Terracotta](https://terracotta.education), an experimental research tool that integrates with a learning management system (LMS) to embed rigorous and responsible research studies in LMS course sites. For additional details, read the preprint. # Materials # Content to be added. # Data # Raw data are exported from Terracotta as a zip archive. The [original zipped export file](https://osf.io/7uz9p) is provided in the "Data" folder. However, to enable more casual viewing of the contents of the export zip, we also provide the individual files contained in the export. The individual files are redundant; they are identical to those in the zip archive (listed here in alphabetical order): - [events.json](https://osf.io/v3zfb) - [experiment.csv](https://osf.io/v2bsj) - [item_responses.csv](https://osf.io/bncf2) - [items.csv](https://osf.io/bncf2) - [outcomes.csv](https://osf.io/wzmka) - [participants.csv](https://osf.io/sjndu) - [participant_treatment.csv](https://osf.io/q87es) - [README.txt](https://osf.io/dvmf9) - [response_options.csv](https://osf.io/9adms) - [submissions.csv](https://osf.io/ey7sj) The [Terracotta Data Dictionary](https://www.terracotta.education/help-center/data-dictionary) provides a detailed account of the format and contents of each of these files. # Analysis # The "Analysis" folder contains the following files: - [CompleteAnalysis.R](https://osf.io/jm5r4) - The R script which implements all analyses in the Terracotta Demonstration Study, and produces summary statistics and figures as presented in the manuscript. The script accesses the data files directly from OSF, as well as a set of helper functions ([DBDA2E-utilities.R](https://osf.io/6v3yg)) which are saved in the preregistration's file storage (see below). This script has three conditional steps, which are currently set to FALSE, which determine whether the script will re-estimate model parameters using MCMC (see comment "# If we're going to rerun the model"). When set to TRUE, the script will produce the following MCMC samples and overwrite them (if they exist) in the current working directory. When set to FALSE, the script will attempt load these samples from the current working directory. - [CodaSamples_Durations.Rdata](https://osf.io/c74w3) - Coda samples for the parameters when modeling the duration spent on learning activities in Terracotta. - [CodaSamples_Scores.Rdata](https://osf.io/3ag27) - Coda samples for the parameters when modeling the scores that participants received on learning activities in Terracotta. - [CodaSamples.Rdata](https://osf.io/het2w) - Coda samples for the parameters when modeling posttest scores. I would've called this "CodaSamples_Posttest.Rdata," except that this filename was included in our registered analysis plan, and I didn't want to deviate from that. - [Model Specification.pdf](https://osf.io/b7cwa) - Narrative description of the analytical models used in [CompleteAnalysis.R](https://osf.io/jm5r4). # Registration # This study was publicly registered prior to data collection on this project site. An initial registration ([December 26, 2021](https://osf.io/wr4tp)) was posted, but we encountered a bug during data collection under that protocol. We then revised the study and posted a new registration ([October 16, 2022](https://osf.io/juq7n)), which included a detailed explanation and justification for the revision, in response to the ["Other" section of the registration questionnaire](https://osf.io/juq7n#61950fb10e2f500009c358da). We then started data collection from scratch. For simplicity and consistency, all contents of this project site reflect data collection under this revised protocol ([October 16, 2021](https://osf.io/juq7n)).