Determining the number of raters for reliable mean ratings

Date created: | Last Updated:


Creating DOI. Please wait...

Create DOI

Category: Project

Description: Script to calculate Cronbach's alphas and correlations with canonical mean ratings for sub-samples with different rater Ns.


How many raters you need to sample to get reliable stimulus ratings will obviously depend on the raters, stimuli and what they’re being rated for. For example, if there is a lot of inter-rater variation or very little inter-stimulus variation, you will need more raters to generate mean ratings with any reliability. If you have a large set of ratings of a type of stimulus, population of rater, and ...


Loading files...



Recent Activity

Loading logs...

OSF does not support the use of Internet Explorer. For optimal performance, please switch to another browser.
This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.