Which statistic measures agreement among judges and ranges from 0 to 1?

Which statistic measures agreement among judges and ranges from 0 to 1?

• A. Kendall's W
• B. Pearson correlation
• C. Spearman's rho
• D. Kendall's tau

Answer: (A)

Agreement among multiple judges is captured by Kendall's W, the coefficient of concordance. It measures how consistently several raters rank the same items and is designed for ordinal data evaluated by more than two judges. The value ranges from 0 to 1, where 0 means no agreement beyond chance and 1 indicates complete agreement among all judges on all items. This makes Kendall's W the right choice for assessing inter-rater reliability across many raters. Pearson correlation, by contrast, looks at linear relationships between two continuous variables and can be negative; it isn’t an inter-rater agreement measure. Spearman's rho and Kendall's tau assess rank-order association between two variables (two sets of rankings) and also range from -1 to 1, not 0 to 1. They’re not suited for evaluating agreement across more than two raters.

Agreement among multiple judges is captured by Kendall's W, the coefficient of concordance. It measures how consistently several raters rank the same items and is designed for ordinal data evaluated by more than two judges. The value ranges from 0 to 1, where 0 means no agreement beyond chance and 1 indicates complete agreement among all judges on all items. This makes Kendall's W the right choice for assessing inter-rater reliability across many raters.

Pearson correlation, by contrast, looks at linear relationships between two continuous variables and can be negative; it isn’t an inter-rater agreement measure. Spearman's rho and Kendall's tau assess rank-order association between two variables (two sets of rankings) and also range from -1 to 1, not 0 to 1. They’re not suited for evaluating agreement across more than two raters.