Date created: | Last Updated:

Category: Project

Description: This is a supplemental material for the paper "The new approach to describe reliability in tests composed of two items only: Admissible and Plausible reliability ranges" Sometimes, researchers want to estimate the test reliability, yet only two items (or subscores) are available. In such cases, a congeneric measurement model (with different linear relations of the items to the true score) is not identified, and thus, the unbiased reliability estimates (such as omega coefficients) cannot be used. We reviewed five conventional approaches under the Classical Test Theory (CTT) geared for such cases, and concluded that all of them pose several assumptions (including tau-equivalent or parallel items and/or apriori known item lengths). We explain how these strong assumptions can bias reliability estimates, especially in tests with different item lengths. Moreover, in specific cases, certain estimates are clearly unrealistic given the possible true reliability values. Further in the paper, we investigate possible bounds of the true reliability given observed data, and suggest using a range in which the reliability parameter shall be located (admissible reliability range), or in which it should be located under very realistic conditions (plausible reliability range). We support the interpretation by a simulation study. Finally, we argue for the new approach (possibly supplemented by the Angoff-Feldt coefficient as a point estimate), and provide recommendations on how to report reliability in tests composed of two items only.