Category kappas for agreement between fuzzy classifications

• Fuzzy kappas for quantifying agreement on a particular category are defined.
• The fuzzy kappa of Dou et al. is a weighted average of the proposed category kappas.
• With two categories the category kappas and overall kappa coincide.
• Adding all other membership functions presents an alternative definition.

The kappa statistic is a widely used as a measure for quantifying agreement between two nominal classifications. The statistic has been extended to the case of two normalized fuzzy classifications. In this paper we define category kappas for quantifying agreement on a particular category of two normalized fuzzy classifications. The overall fuzzy kappa is a weighted average of the proposed category kappas. Since the value of the overall kappa lies between the minimum and maximum values of the category kappas, the overall kappa, in a way, summarizes the agreement reflected in the category kappas. The overall kappa meaningfully reflects the degree of agreement between the fuzzy classifications if the category kappas are approximately equal. If this is not the case, it is more informative to report the category kappas.