We will exclude participants who used the same rating for all the names within at least one scenario.
We will perform the analysis using mixed-effect models with the rating of riskiness as the dependent variable, random intercepts for words and participants, and word length, scenario, scenario order, and pronounceability as predictors. We will also include the interaction between pronounceability and scenario, the interaction between scenario and scenario order, and the interaction between pronounceability and word length as predictors. We will include random slopes for participants for pronounceability, word length, and scenario.
We are mostly interested in the existence of the effect in general, so we will perform the primary analysis using only the new names. We suspect that testing the interaction between pronounceability and the source of names (old vs. new) would have low statistical power due to the low number of names in the Song and Schwarz (2009) study, a problem which is further exacerbated because pronounceability is confounded with the name length in the names used in the original study. However, we will also perform an analysis only for the old names and an analysis with the source of names as a predictor.
We tested the analysis on simulated data. An R script for the data simulation and analysis on the simulated data can be found in Files.