The following repository contains all analysis scripts, datasets, and supplementary materials to two articles exploring reliability and validity in the context of two-choice reaction time tasks, particularly the Approach-Avoidance Task (AAT). As this is an ongoing project, files in the repository may change, or new files may be added as the project progresses. The following is a description of all files included so far. ----- The following files belong to both articles: simulation.plots.R - R script containing code for generating all plots in the papers pertaining to results derived from simulations ----- The following files belong to the article listed below: Kahveci, S., Rinck, M., van Alebeek, H., & Blechert, J. (2023). How pre-processing decisions affect the reliability and validity of the approach–avoidance task: Evidence from simulations and multiverse analyses with six datasets. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02109-1 Study 1: ReviewDatabase.xlsx - excel sheet containing information on all studies reviewed for the literature review included within the manuscripts associated with this repository. review.summarizer.R - R script used to derive count data from the aforementioned literature review Study 2: simulation.knockout.R - R script for analyzing how the reliability of double difference scores, compatibility scores, and multilevel bias scores changes with the removal of an increasing number of trials from a specific condition simulation.knockout.csv - results of the synonymous analysis script Study 3: simulation.newoutliers.R - R script for analyzing how reliability of simulated data is affected by outliers and how different scoring algorithms and outlier rejection methods influence these changes in reliability simulation.newoutliers.csv - results of the synonymous analysis script, specifically the effect of outlier rejection and scoring algorithms on reliability simulation.newoldetection.csv - results of simulation.newoutliers.R, specifically the sensitivity and specificity of different outlier rejection methods Study 4: alldata.rda - the 6 empirical AAT datasets used in studies analyses.descriptives.R - R script containing code used to extract the properties of the 6 datasets, and generate the associated table depicted in the manuscript. analyses.datastats.csv - data file containing relevant properties of the 6 empirical datasets analyses.computations.new.R - R script for computing a multiverse analysis of pre-processing decisions applied to 6 empirical AAT datasets analyses.computations.new.*.csv - results of the synonymous analysis script analyses.idealiter.R - R script for analyzing the ideal number of iterations for computing split-half reliability. analyses.idealiter.csv - results of the synonymous analysis script All studies: analyses.plots.R - R script containing code for generating all plots in the papers pertaining to results derived from empirical data ----- The following files belong to the article listed below: Kahveci, S., Bathke, A.C., Blechert, J. (2022). Reliability of reaction time tasks: how should it be computed?. PsyArXiv. https://doi.org/10.31234/osf.io/ta59r Study 1: simulation.relcompare.R - R script for analyzing the accuracy of different reliability computation algorithms simulation.relcompare.*.csv - csv file with results of the synonymous analysis script Study 2: simulation.idealiter.R - R script for analyzing the ideal number of split-half iterations in simulated data with varying properties simulation.idealiter4.*.rda - R data files containing results of the synonymous analysis script Study 3: simulation.confint.R - R script for analyzing the effect of autocorrelation and the accuracy of the 95% confidence intervals simulation.confint.*.csv - csv file with results of the synonymous analysis script Study 4: reloverview.xlsx - Excel document listing Cronbach's alpha and split-half reliabilities from reaction time studies as part of a meta-analysis on the difference in magnitude between these two analyses.reloverview.R - R script used to compute some of the above reliabilities (for studies from which the data was available); the reliabilities were also plotted and analyzed with this script. Note that the original datasets for these studies are available from the original authors, see the manuscript. Appendix: simulation.packcomp.R - R script for analyzing the accuracy of different permutation-based split-half reliability coefficients from different R packages simulation.packcomp.*.csv - csv file with results of the synonymous analysis script Additional: Exploratory findings on autocorrelation.docx - Document discussing additional findings on autocorrelation not included in the main manuscript Kind regards, Sercan Kahveci