Date created: | Last Updated:

Category: Project

Description: This OSF project is associated with a study that evaluates a range of keyness metrics in terms of their performance and reliability. The paper delineates four dimensions of keyness, which distinguish between frequency- and dispersion-related perspectives. Existing keyness metrics are then organized according to these dimensions and evaluated with regard to their performance on a specific keyword analysis task: The identification of key verbs in academic writing. To this end, the rankings produced by 32 different metrics are evaluated against an established academic word list. Further, the reliability of metrics is measured, to determine whether they produce stable and replicable rankings across repeated studies on the same pair of text varieties. We observe notable differences among metrics with regard to these criteria. Our findings provide further support for the superiority of the Wilcoxon rank sum test and text-dispersion-based measures, and allow us to identify, within each dimension of keyness, metrics that may, in light of our criteria, be given preference in applied work.