Main content
Determining the number of raters for reliable mean ratings
Date created: | Last Updated:
: DOI | ARK
Creating DOI. Please wait...
Category: Project
Description: Script to calculate Cronbach's alphas and correlations with canonical mean ratings for sub-samples with different rater Ns.
How many raters you need to sample to get reliable stimulus ratings will obviously depend on the raters, stimuli and what they’re being rated for. For example, if there is a lot of inter-rater variation or very little inter-stimulus variation, you will need more raters to generate mean ratings with any reliability.
If you have a large set of ratings of a type of stimulus, population of rater, and …