Quantifying idiosyncratic and shared contributions to stimulus evaluations

Contributors:
  1. Alexander Todorov

Date created: | Last Updated:

: DOI | ARK

Creating DOI. Please wait...

Create DOI

Category: Project

Description: Although identifying the idiosyncratic and shared contributions to stimulus evaluation is essential for many psychological domains, there is no established method for estimating these contributions. To examine different estimation methods (Study 1), participants rated the beauty of stimuli varying in within-person reliability and between-rater agreement: faces (high on both), objects (high on the former, low on the latter), and complex patterns (low on both). Variance component analyses (VCAs) better captured the psychometric properties of the data than other published metrics. We further examined how the VCAs estimates are affected by the number of repeated measurements and data preprocessing (Study 2). VCAs stabilized with more measures, consistent with the finding that both within-person reliability and between-rater agreement increased nonlinearly with repeated measures. Our findings suggest that researchers should collect more than two repeated measures to assess idiosyncratic versus shared contributions to stimulus evaluation. Mixed models with crossed random effects are currently the best way to render these estimates most reliable.

License: CC-By Attribution 4.0 International

This project represents an accepted preprint submitted to PsyArXiv . Learn more about how to work with preprint files. View preprint

Files

Loading files...

Citation

Tags

Recent Activity

Loading logs...

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.
Accept
×

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Create an Account Learn More Hide this message