Quantifying idiosyncratic and shared contributions to stimulus evaluations

  1. Alexander Todorov

Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Project

Description: Although identifying the idiosyncratic and shared contributions to stimulus evaluation is essential for many psychological domains, there is no established method for estimating these contributions. To examine different estimation methods (Study 1), participants rated the beauty of stimuli varying in within-person reliability and between-rater agreement: faces (high on both), objects (high on the former, low on the latter), and complex patterns (low on both). Variance component analyses (VCAs) better captured the psychometric properties of the data than other published metrics. We further examined how the VCAs estimates are affected by the number of repeated measurements and data preprocessing (Study 2). VCAs stabilized with more measures, consistent with the finding that both within-person reliability and between-rater agreement increased nonlinearly with repeated measures. Our findings suggest that researchers should collect more than two repeated measures to assess idiosyncratic versus shared contributions to stimulus evaluation. Mixed models with crossed random effects are currently the best way to render these estimates most reliable.

License: CC-By Attribution 4.0 International


Loading files...



Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.