Fuzzy kappa for the agreement measure of fuzzy classifications

In this paper, we propose an assessment method of agreement between fuzzy sets, called fuzzy Kappa which is deduced from the concept of Cohen's Kappa statistic. In fuzzy case, the Cohen's Kappa coefficient can be calculated generally by transforming the fuzzy sets into some crisp αα-cut subsets. While the proposed fuzzy Kappa allows to directly evaluate an overall agreement between two fuzzy sets. Hence, it is an efficient agreement measure between a given fuzzy “ground truth” or reference and a result of fuzzy classification or fuzzy segmentation. Based on membership function, we define its agreement function and its probability distribution to formulate the deduction of the expectation agreement. So the fuzzy Kappa is calculated from the proportion of the observed agreement and the agreement expected by chance. All the definitions and deductions are detailed in this paper. Both Cohen's Kappa and the fuzzy Kappa are then used to evaluate the agreement between a fuzzy classification of brain tissues on MRI images and its “ground truth”. A comparison of the two types of Kappa coefficient is carried out and shows the advantage of the fuzzy Kappa and some limitations of Cohen's Kappa in the fuzzy case.