The statistical analyses in nearly all of the published articles that evaluated law enforcement procedures procedures (e.g., sequential versus simultaneous lineups), or effects of "uncontrollable" factors (e.g., presence or absence of weapon) generally looked at only one aspect at a time and relied on either (a) diagnostic ratio (DR = HR/FAR = hit rate / false alarm rate, or Sensitivity / (1 – Specificity)), or (b) Expressed Confidence Level (ECL)-based ROC curve (i.e., a plot of hit rate versus false alarm rate, for different levels of the confidence expressed by “eyewitnesses,” usually participants in EWI studies). The ECL-based ROC may be viewed as an approach to account for a single covariate (in this case, apparent confidence) on accuracy. While the use of covariates in assessing EWI accuracy is sensible, other appropriate and relevant covariates should be considered, some of which reflect eyewitness confidence or response bias: ECL may, or may not, be the best measure for this effect (see #3). In addition, other statistical approaches that properly incorporate several covariates will be more powerful and should be considered. Some examples are: • Statistical model of accuracy using regression (linear model for logit(HR), logit(FAR), or log(DR)); • Bivariate statistical models that involve both DR, called LR+ in the medical diagnostic literature, and LR−. LR+/LR− are related to Positive/Negative Predictive Value (PPV/NPV); both are critical to law enforcement. Thus, statistical models must account for variables that affect both outcomes, LR+ and LR−. Eusebi et al. (2014) develop a latent class bivariate model for sensitivity and specificity jointly, and Wang and Gatsonis (2008) propose a predic- tive receiver operating characteristic (PROC) that incorporates both PPV and NPV; both will be investigated and others will be developed for EWI studies. • Binary classification: In effect, each “eyewitness” is a binary classifier, and studies of EWI involve many such “binary classifiers.” The literature in statistics and machine learning is rich in publications on combinations of classifiers, which can be considered here for more powerful analyses of different EWI procedures. For example, Keith et al. (2012) use a Bayesian hierarchical model to combine classifiers. Other models, such as those suggested in Hastie et al. (2009), will be investigated, and new ones developed. • Control versus Noise variables: In the 1980s, Genechi Taguchi recommended analyses of production experiments that focused on optimal values of “control” variables (those under operator control, such as instrument settings) that minimize the effects of “noise” variables (e.g., temperature variations); see Nair et al. 1992. The same concept can be applied here with system and estimator variables. The analysis of data from fractional factorial experiments (cf.#3) involves both system and estimator variables from the perspective of identifying those settings of the system (control) variables that both maximize the response (sensitivity and specificity) and are least sensitive to settings of estimator variables (over which we have no control). Alternatively, the research may suggest certain levels of estimator variables (e.g., presence of weapon and poor lightening) that dictate one form of lineup over another. Starting with lineup studies and using the results of the review in #1 above, we will apply these more powerful approaches to the data in those studies that show the most promising effects. Eusebi P, Reitsma JB Vermunt JK (2014), Latent class bivariate model for the meta-analysis of diagnostic test accuracy studies. BMC Medical Research Methodology 14:88 (http://www.biomedcentral.com/1471-2288/14/88). Hastie T; Tibshirani R, Friedman JH (2009), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition, Springer (http://statweb.stanford.edu/∼tibs/ElemStatLearn). Keith JM, Davey CM, Boyd SE (2012), A Bayesian method for comparing and combining binary classifiers in the absence of a gold standard. BMC Bioinformatics 13:179 (doi:10.1186/1471-2105-13-179). Nair VN (1992), Taguchi’s parameter design: A panel discussion. Technometrics 34:127161 (doi:10.1080/00401706.1992.10484904). Wang F; Gatsonis, CA (2008) Hierarchical models for ROC curve summary measures: Design and analysis of multi-reader, multi-modality studies of medical tests", Statistics in Medicine 27:243-256 (doi: 10.1002/sim.2828)