This project includes simulation results and code for our paper on non-normal random effects (inferring whether there is more than one group of subjects or items).
The code is in the Scripts folder
The workedExample.Rmd notebook is the code to examine for most users. It shows how to estimate the number of groups in your data, cluster subjects into groups, and estimate false alarm rate and power for various methods of estimating normality and bimodality
mainSimulation.R is the script we used to generate the simulation results in the Output of Simulations folder. The filenames in that folder indicate whether the distribution of responses was balanced or skewed, the size of the group effects, underlying bimodality or unimodality, the number of subjects, the denominator by which the subject and item standard deviations from our original study was divided (e.g., sd3 is original by-subj sd divided by 3), number of items and probability of a subject being assigned to group 1 (.63 opr .8). More simulations of this sort can be generated using this script by changing these parameters in the script.
aggregatingSimulationResults.R takes in the output of simulation, as exemplified by the files in Example Outputs Of Simulations and applies all of the normality and modality tests we evaluated to it. This is the script we used to evaluate these tests
aggregatingOnlyBIC.R was used to calculate BICs for models with and without the effect of group method
analysis.R is the script we used to analyze the simulation results, as well as to generate the figures in the Figures folder, some of which are in the paper
The Empirical Study Dataset folder contains the dataset from our empirical study and analysis code for it. This dataset has 'hopeless' quasi-separation and exemplifies data for which the methods proposed here fail to determine the number of underlying groups